ModelRefs / Contextual Retrieval — AI Glossary

Contextual Retrieval — AI Glossary

An Anthropic-developed RAG enhancement that prepends chunk-specific context (generated by an LLM) before embedding each chunk.

Overview

Contextual retrieval significantly reduces retrieval failures by adding document context that would otherwise be lost when chunks are split. Combined with hybrid search and reranking, it reduces top-20 retrieval failures by ~70% vs naive RAG.

Reference details

Topic	rag
Last reviewed	2026-06-24

How it works

Before embedding, each chunk is passed to a model along with the whole document, and the model writes a short passage situating that chunk within it.
The generated context is prepended to the chunk, and the combined text is what gets embedded — so the stored vector reflects both the chunk's content and where it sits.
This addresses a specific failure of flat chunking: a chunk saying 'the limit was raised to 50' loses the subject, the product and the date that the surrounding document supplied, and no embedding of that sentence alone can match a query naming them.
The chunk text itself is also indexed for lexical search, so exact terms remain findable and the added context improves the semantic side without diluting the keyword side.
It composes with hybrid search and reranking rather than replacing them, and the reported gains are for the combination rather than for contextualisation alone.
The cost is one model call per chunk with the full document in the prompt, which is what prompt caching exists to make affordable.

Example: Why prompt caching is what makes this affordable

Take a 100,000-token document split into 10,000 chunks across a corpus. Contextualising each chunk means sending the document with every call, so the input is 10,000 x 100,000 = 1,000,000,000 tokens — a billion, which at 3 units per million is 1,000 x 3 = 3,000 units for one corpus, and the document has been read ten thousand times. With prompt caching on a stable prefix billed at a tenth of the input rate, the same work costs about 3,000 / 10 = 300 units. That is the difference between a technique that is interesting in a paper and one that is deployable: 3,000 units per corpus refresh is a real line item, and 300 is not. The arithmetic also sets the design constraint — the document must be the stable prefix and the chunk the variable suffix, in that order, or the cache does not hit and the cost reverts to the full 3,000. Getting the prompt order backwards is the common implementation error, and it shows up as a bill rather than as a wrong answer.

Commonly confused with

This is a change to what gets embedded, not a change to retrieval or ranking, which is why it composes with everything downstream. Query expansion generates variants of the question; reranking reorders candidates after retrieval; a hierarchical index adds summary nodes above the chunks. Contextualisation modifies the chunk's own representation before it is indexed, so it improves the first-stage recall ceiling that every later stage is bounded by. The practical implication is ordering: fixing the chunk representation raises the ceiling, and adding a reranker under a low ceiling does not.

When to use it

Reach for it when:

Corpora where chunks lose essential context when split — documents with running subjects, pronouns, or implicit dates.
With prompt caching available, which is what makes the per-chunk contextualisation cost tolerable.
Alongside hybrid search and reranking, since the reported gains are for the combination.
Where first-stage recall is the measured bottleneck, which is the thing this addresses.

Reach for something else when:

Self-contained chunks — reference entries, logs, independent records — where no document context is being lost.
Without prompt caching, where the full-document-per-chunk cost is an order of magnitude higher.
Frequently changing corpora, since every change means regenerating context and re-embedding.
Before measuring retrieval recall, which determines whether the first stage is actually the constraint.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Contextual Retrieval — AI Glossary.

Frequently asked questions

What is Contextual Retrieval?