Domain Adaptation — AI Glossary

Reference details

Related terms

• Transfer Learning
• Continual Learning
• Supervised Fine-Tuning (SFT)

How it works

1. Continued pretraining runs the next-token objective over a domain corpus, before any instruction tuning.
2. That stage shifts the model's distribution toward domain vocabulary and phrasing, which is what retrieval cannot do since retrieval changes the prompt rather than the model.
3. Mixing in general data during the run limits how much general capability is traded away for domain gain.
4. Instruction tuning is then redone or reapplied, because continued pretraining on raw text degrades the assistant behaviour it was layered over.
5. Evaluation must cover general capability as well as domain tasks, since the failure mode is a model that improved in-domain and regressed everywhere else.

Example: Why the data requirement is the gate

Continued pretraining typically wants on the order of billions of domain tokens to shift a model's distribution meaningfully. A specialist corpus of 10,000 documents at 5,000 tokens each is 50 million tokens — well under 1% of that, and will mostly produce overfitting rather than adaptation. The same 50 million tokens used as a retrieval corpus is substantial, covering the material directly at inference. The data volume, not the technique, usually decides which approach is available.

Commonly confused with

Conflated with fine-tuning and with RAG, and it differs from both in what it changes. Instruction fine-tuning teaches behaviour from input-output pairs; this changes the model's language distribution from raw domain text. RAG changes neither and supplies knowledge at inference. The practical rule is that retrieval handles facts, fine-tuning handles format and behaviour, and domain adaptation handles vocabulary and register — the case where the model reads the domain's text poorly rather than lacking its facts.

When to use it

Reach for it when:

• The domain's vocabulary and phrasing are genuinely foreign to the base model.
• Billions of domain tokens are available, which the technique requires to work.
• Retrieval has been tried and the failure is comprehension rather than missing facts.

Reach for something else when:

• The gap is facts, where retrieval is cheaper and carries no regression risk.
• Domain data is thousands of documents rather than billions of tokens.
• General capability is not being evaluated, so a trade-off will go unnoticed.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Domain Adaptation — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path