ModelRefs / Positional Bias — AI Glossary

Positional Bias — AI Glossary

A model's tendency to favor answer options at specific positions (e.g., first, last) independent of content quality.

Overview

Positional bias affects multiple-choice QA (models prefer option A or the last option) and long-context retrieval (primacy/recency effects). Can inflate benchmark scores if tests are not position-balanced. Mitigated by ensembling across position permutations or using calibration techniques.

Reference details

Topic	prompting
Last reviewed	2026-06-24

How it works

Attention over a long sequence is not uniform, so identical content is weighted differently depending on where it sits.
The effect concentrates at the ends: material at the start and the end of a context is recalled more reliably than material in the middle.
In multiple-choice formats this appears as a preference for particular option positions, independent of the option's content.
That makes any unbalanced test measurable but not meaningful — a score can reflect option ordering rather than knowledge.
The mitigation is to vary position and aggregate: shuffle options across runs, or place the same evidence at different depths and compare.

Example: How position inflates a benchmark score

A four-option test where the correct answer sits at position A in 40% of items, against the 25% a balanced set would give. A model with a mild preference for the first option gains on every one of those items for reasons unrelated to knowing the answer. Shuffling positions and rerunning separates the two effects: if the score falls, the gap was positional. The test was not wrong to measure, only wrong to attribute — and nothing in the aggregate figure reveals which it was.

Commonly confused with

Positional bias is the general effect — content is judged partly by where it sits. Recency bias is one specific case, the tail of the context being over-weighted; primacy is the other end; lost-in-the-middle is the observation that both ends beat the middle. Naming the specific position matters, because the mitigation differs: shuffling option order addresses one, reordering retrieved context addresses another.

When to use it

Reach for it when:

Designing or auditing multiple-choice evaluations, where option order must be balanced or shuffled.
Placing critical instructions or evidence in a long prompt, where the middle is the weakest position.
Explaining why a model recalls the top and bottom of a document better than the centre.

Reach for something else when:

Short prompts, where there is little positional range for the effect to act over.
As a synonym for recency bias, which is the specific case of the tail dominating.
Explaining a wrong answer with no positional variation tested — that is an assertion, not a diagnosis.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Positional Bias — AI Glossary.

Frequently asked questions

What is Positional Bias?