Base Model — AI Glossary

Reference details

Related terms

• Large Language Model
• Pretraining
• Instruction-Tuned Model
• Supervised Fine-Tuning (SFT)

How it works

1. Pretraining optimises one objective over a large corpus: predict the next token. Nothing in it teaches the model that a prompt is a request to be answered.
2. The result is a distribution over plausible continuations of text, which is why a question can be continued with more questions rather than an answer.
3. Instruction tuning then fits the model to request-response pairs, teaching the format of being asked and answering.
4. A preference stage — RLHF or a direct method such as DPO — tunes which of several acceptable answers is preferred, adding refusal behaviour and tone.
5. Each stage narrows the output distribution, which is why a base model is the better starting point for a specialised fine-tune: there is less prior behaviour to overwrite.

Example: The same prompt, before and after instruction tuning

Give a base model 'What is the capital of France?' and a well-formed continuation is another line of quiz questions, because the corpus contains many such lists and the model is completing a document. The instruction-tuned sibling answers 'Paris'. Identical weights up to the tuning stage, identical prompt, different behaviour — and the base model was not wrong about the text, only about what was wanted. This is the surprise worth internalising before benchmarking a base checkpoint as an assistant.

Commonly confused with

Base means pretrained and nothing more — no instruction tuning, no alignment. This produces a specific and much-repeated surprise: prompt a base model with a question and it may continue the text with more questions, because it was never trained to treat a prompt as a request. Instruction-tuned and chat variants of the same weights behave completely differently and are usually what people actually want.

When to use it

Reach for it when:

• Fine-tuning for a specialised behaviour, where prior instruction tuning would have to be partly undone.
• Research into pretraining behaviour, where alignment effects are a confound.
• Building a non-chat completion product such as code infilling.

Reach for something else when:

• Anything that should answer requests out of the box — reach for the instruct variant.
• Benchmarking assistant quality, where scores will reflect the missing tuning rather than the model.
• User-facing deployment without a safety stage, since refusal behaviour comes from training the base model has not had.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Base Model — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path