Transformer — AI Glossary

Reference details

Related terms

• Attention Mechanism
• Large Language Model
• Embedding
• KV Cache

How it works

1. Tokens are mapped to embedding vectors and combined with positional information, which is necessary because attention itself is order-agnostic.
2. The stack is then N identical blocks, and depth is the only thing that differs between a small model and a large one of the same design.
3. Each block normalises its input, computes multi-head self-attention, and adds the result back to the input it received rather than replacing it.
4. It then normalises again and applies a position-wise feed-forward network that expands to a wider dimension and contracts back, which is where most of the parameters sit.
5. The additions around both sub-layers are residual connections, and they are what makes a deep stack trainable at all: gradients reach early layers through the additions rather than through every transformation.
6. After the final block a last normalisation and a projection to vocabulary size produce the logits, and in a decoder-only model that projection often shares weights with the input embedding.

Example: What self-attention buys, and the parallelism that actually won

In “the trophy did not fit in the suitcase because it was too small”, resolving “it” requires weighing every other word. Recurrent models pass information along a chain and degrade with distance; self-attention lets every token attend to every other directly, in one step. The cost is that attention is quadratic in sequence length, which is why long context is expensive. The property that decided the architecture's adoption, though, was training throughput rather than accuracy. A recurrent model processing a 1,000-token sequence must take 1,000 sequential steps, because position n cannot be computed before position n-1. A transformer computes all 1,000 positions in parallel within each layer, so a 32-layer stack needs 32 sequential steps rather than 1,000 — a reduction of 1,000 / 32 = 31.25 in the depth of the dependency chain. That is what made training on web-scale corpora feasible at all.

Commonly confused with

Transformer is the architecture, not the model. GPT, Claude, Llama and BERT are all transformers but differ in pretraining, scale and whether they are decoder-only, encoder-only or encoder-decoder. Saying a system “uses a transformer” says almost nothing about what it can do. The parallelism is also a training property rather than an inference one: generation is still sequential because each token conditions on the last, so the architecture removed the dependency chain from training and left it in place for decode.

When to use it

Reach for it when:

• It is the default for sequence modelling; the real choice is which variant and size
• Long-range dependencies matter and recurrence would lose them
• You need parallel training over sequences rather than step-by-step
• When explaining why the architecture displaced recurrence, which was throughput before accuracy

Reach for something else when:

• Sequences are extremely long and quadratic attention dominates — state-space models target this
• The task is tabular or small-data, where simpler models win on cost and interpretability
• Expecting the training parallelism to speed up generation, which remains sequential per token
• As a description of capability, since every major model family shares the architecture

Primary source

• Vaswani et al., Attention Is All You Need (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Transformer — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path