Lost in the Middle — AI Glossary

Reference details

Related terms

• Context Stuffing
• Long-Context Model
• Positional Bias

How it works

1. Two separate effects give attention its shape across a long context, and the middle is where neither applies.
2. The first tokens of a sequence receive disproportionate attention regardless of their content, because the softmax must distribute weight somewhere and early positions are visible to every later query.
3. That makes the opening of a context structurally privileged — a property of the mechanism rather than of what was placed there.
4. The second effect is recency: under a causal mask the most recent tokens have had the fewest competing positions to share attention with, and relative position schemes additionally bias toward small offsets.
5. Material in the middle is neither structurally privileged nor recent, so it competes for attention on content alone against everything else in the window.
6. The practical consequence is that position is a parameter of a retrieval-augmented prompt in the same way the chunk count is, and the ordering of retrieved passages is therefore a tuning decision rather than an arbitrary one.

Example: Position changes the answer

Place the answer-bearing passage first among twenty and the model finds it reliably. Move the same passage to position ten and accuracy drops measurably, with no change to the passage or the question. Retrieval accuracy follows a U across position — strong at both ends, weakest in the middle.

Commonly confused with

This is not a context-length limit. The model accepts the tokens and processes them; it attends to the middle less effectively. A page that fits comfortably inside the window can still be read poorly, which is why fitting is not the same as using.

When to use it

Reach for it when:

• Ordering retrieved chunks — put the strongest first or last
• Explaining why long-context recall tests disappoint in practice
• Deciding between fewer, better passages and more, worse ones

Reach for something else when:

• As an argument against long context generally — the effect varies by model
• Assuming it is fixed; newer models differ and it is worth re-testing

Primary source

• Liu et al., Lost in the Middle: How Language Models Use Long Contexts (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Lost in the Middle — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path