Supervised Fine-Tuning (SFT) — AI Glossary

Reference details

Related terms

• Fine-Tuning
• RLHF (Reinforcement Learning from Human Feedback)
• Direct Preference Optimization (DPO)
• Base Model

How it works

1. Each instruction-response pair is rendered into the model's chat template, so the training text has exactly the delimiter structure that inference will later produce.
2. The loss is masked so that only the response tokens contribute: the prompt is present as context but is not a prediction target.
3. That masking is what separates instruction tuning from continued pretraining — without it the model learns to generate plausible instructions as readily as plausible answers.
4. Short examples are packed several to a sequence to avoid wasting positions on padding, with attention masked at the boundaries so one example cannot attend into the next.
5. Training runs for very few epochs at a low learning rate, because the dataset is small relative to pretraining and the model fits it fast enough that overfitting arrives within one or two passes.
6. A held-out split is scored throughout, since the failure here is not divergence but a model that reproduces the training set's quirks — including any inconsistency in how the examples were written.

Example: What the training pair looks like

Not documents, but demonstrations: an instruction paired with the response you want. “Summarise this ticket in two sentences, ending with the next action.” → the ideal two-sentence summary. Ten thousand of those teach form and register. Feeding raw documents instead teaches the model to continue documents, which is not what you asked for.

Commonly confused with

SFT is not pretraining and not alignment. Pretraining learns language from raw text; SFT teaches instruction-following from curated pairs; RLHF or DPO then tunes preferences between acceptable responses. SFT sets the baseline behaviour the preference stage refines.

When to use it

Reach for it when:

• You need consistent instruction-following in a specific format or domain
• You have curated examples of the task done correctly
• It is the prerequisite step before any preference-based alignment

Reach for something else when:

• Your examples are inconsistent — SFT will learn the inconsistency faithfully
• The goal is factual knowledge rather than behaviour
• Prompting has not been exhausted; it is reversible and SFT is not

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Supervised Fine-Tuning (SFT) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path