Model Architecture — AI Glossary

Reference details

Related terms

• Transformer
• Decoder-Only Architecture
• Encoder-Decoder Architecture

How it works

1. Architecture fixes the structural choices before any training: stack type, attention variant, normalisation placement, activation, and positional encoding.
2. Each choice has a measurable consequence — attention variant sets KV-cache size, activation sets parameter count at a given width, positional encoding sets how far context extrapolates.
3. These choices are largely irreversible after pretraining, because the weights were learned inside them.
4. They determine inference cost as much as capability: two models of identical parameter count can differ severalfold in cache footprint and therefore in achievable concurrency.
5. Published architectures converge because the trade-offs are empirical, which is why current frontier models look structurally similar.

Example: Why the attention variant decides concurrency

At 32 attention heads, a full multi-head cache stores keys and values for all 32. Grouped-query attention sharing one key-value pair across 8 heads stores 4 sets instead — an eightfold reduction in cache per token. Since concurrent requests are usually capped by cache memory rather than compute, that is roughly eight times the concurrent sequences on the same device. One architectural decision, made before training, sets the serving economics for the model's whole life.

Commonly confused with

Frequently reduced to parameter count, which is a single number describing none of this. Two 7B models can differ in cache footprint, context extrapolation and training stability because of choices parameter count does not express. It is also not the same as model family or size: a family shares an architecture across sizes, and architecture is the shared part. Nor is it configuration — temperature and context length are set per request, while these choices were fixed before pretraining and cannot be changed after.

When to use it

Reach for it when:

• Explaining why two models of similar size differ in serving cost or context behaviour.
• Reasoning about KV-cache footprint, which the attention variant determines.
• Understanding which properties were fixed before training and cannot be tuned later.

Reach for something else when:

• As a synonym for parameter count or model size.
• Describing per-request settings, which are configuration.
• Explaining learned behaviour such as refusal or instruction-following, which comes from training.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Model Architecture — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path