Supervised Fine-Tuning as Behavior Shaping

A training row says which response should become more likely in a context. It also implicitly teaches tone, completeness, tool-use conventions, uncertainty handling, refusal behavior, and when to ask a question. If annotators reward a fluent unsupported answer, the model is trained toward fluent unsupported answers. Start with a behavior rubric and representative failure cases, then author demonstrations that expose the desired decision—not just a polished final sentence.

Separate stable behavior from volatile knowledge. A formatting or domain-language pattern can be demonstrated repeatedly; today's refund limit should generally remain attributable to an authorized current source. Include examples that use evidence, recognize missing evidence, refuse unauthorized requests, and recover from tool errors. Synthetic examples need the same review and provenance discipline as collected examples; a generation model's mistakes do not become truth because they are numerous.

A loss mask answers whether a target token contributes to the objective. An attention mask answers which context positions a token may read. Setting user loss to zero does not remove user context; blocking attention does. Confusing these masks can train a model on the wrong targets or let packed examples read unrelated conversations. Test token IDs, rendered roles, shifted labels, loss masks, and attention visibility together on small hand-inspectable examples before scaling.

Keep whole conversations inside one split. A system message may introduce an instruction, followed by alternating user and assistant turns; a row should not end halfway through the target answer. Truncation can erase the evidence while preserving the answer, change a refusal into compliance, or delete an end-of-turn marker. The teaching contract rejects over-budget conversations instead of silently truncating them. A production pipeline needs an explicit truncation policy with before-and-after behavioral fixtures.

Batching changes which gradients are averaged together and how frequently examples are exposed. Repeated long examples may dominate even when row counts look balanced. Record supervised-token counts, sampling weights, length distributions, and repetition by source. This artifact audits row semantics only; it does not implement a packer or prove attention-mask correctness in a training framework.

A random row split can place adjacent messages from the same conversation, rewritten answers from the same source document, or repeated customer incidents on both sides. Choose a causal grouping unit—conversation, source document, entity, organization, or time window—before building train and evaluation material. Preserve split provenance so an evaluator can explain why a held-out row is genuinely independent of the training path.

The contract rejects reused conversation IDs, repeated tokenized conversation contents, and source groups crossing train/eval. Those are precise checks, not a comprehensive contamination detector. Near duplicates, paraphrases, quoted answers, pretraining contamination, and temporal leakage require additional review and evidence. A content digest binds what was checked; it does not prove that every form of semantic overlap was detected.

The fixture has one synthetic training conversation and one independent synthetic evaluation conversation. Frozen rows contain defensively copied turns and integer token/mask tuples. The public audit reconstructs nested records to catch constructor bypass, binds exact evaluation row contents, and rejects malformed masks, unsupported roles, leakage, unlicensed rows, pending PII review, and unqualified claims. Its two supervised training tokens are accounting facts about this toy fixture, not a recommendation for dataset size.

1def main():
2    result = audit(example_contract())
3    print("example=illustrative_only")
4    print(f"rows=train:{result.train_rows},eval:{result.eval_rows}")
5    print(f"supervised_train_tokens={result.supervised_tokens}")
6    print("claim=" + result.claim)

example=illustrative_only
rows=train:1,eval:1
supervised_train_tokens=2
claim=DATA_CONTRACT_ONLY_NO_GENERALIZATION_CLAIM

A dataset version name is not enough: keep model, tokenizer, policy, source provenance, deduplication procedure, evaluation rubric, and row content in the identity. Recompute the evaluation digest after any authorized evaluation change; do not silently reuse old results. In a production release, bind the dataset identity to the actual training configuration, checkpoint, and evaluation outputs as the next layer of the contract.

Training loss establishes how well optimization fits supervised targets. Held-out token loss establishes predictive fit on the chosen holdout. Neither alone establishes the product outcome. Compare the adapted model with the untuned baseline on task completion, unsupported claims, format validity, abstention, security behavior, and regressions outside the tuned domain. Include new entities, phrasing, source combinations, and boundary cases chosen without inspecting training outputs.

Use an evaluation set for iteration and reserve an independently controlled final gate when repeated tuning would otherwise overfit the benchmark. Inspect cohort-level deltas and uncertainty, not just an aggregate. If training improves while held-out behavior degrades, investigate repeated data, label quality, task mismatch, too much optimization, and template drift. Do not label a memorized response as a learned capability without evidence of transfer.