ModelRefs / Document Summary Index — AI Glossary

Document Summary Index — AI Glossary

A RAG index storing LLM-generated summaries per document alongside full chunks, enabling summary-level retrieval followed by chunk-level reading.

Overview

LlamaIndex's DocumentSummaryIndex generates a summary per document at index time. At query time, the summary index is searched first to identify relevant documents; full chunks of those documents are then used for generation. Improves performance on global questions requiring understanding of entire documents, outperforming flat chunk retrieval.

Reference details

Topic	rag
Last reviewed	2026-06-24

How it works

At index time each document is read in full and a model writes a summary of it, which is embedded and stored alongside the document's identifier.
The document's ordinary chunks are also embedded and stored, but they are keyed by parent document rather than held in one flat pool.
A query first searches only the summary vectors, so the retrieval unit at this stage is a whole document rather than a fragment of one.
The top-ranked documents are selected, and their chunks are then gathered — either all of them or a second ranking within that restricted set.
Generation runs over chunks drawn from documents chosen for whole-document relevance, which is a different selection than ranking every chunk independently.
Because the summary is written once at index time, the per-query cost is a small vector search while the summarisation cost is paid up front.

Example: What two-stage selection narrows, and what it loses

Take 2,000 documents averaging 40 chunks each, so 80,000 chunks in total. Flat retrieval ranks all 80,000 vectors and returns the top handful, which means a document's relevance is only ever expressed through whichever of its fragments happens to rank. A summary index searches 2,000 summary vectors instead, selects the top 5 documents, and then considers 5 x 40 = 200 chunks — so the second stage reads from a pool of 200 rather than 80,000, a narrowing of 80,000 / 200 = 400 to 1. The index-time bill is modest and one-off: summarising 2,000 documents at 6,000 input and 300 output tokens each is 12,000,000 input and 600,000 output tokens, which at $3 and $15 per million is 12 x $3 = $36 plus 0.6 x $15 = $9, so $45. The loss is in the same arithmetic. A 300-token summary of a 6,000-token document is a compression of 6,000 / 300 = 20 to 1, and anything the summary omits is invisible to the first stage. So a question about a document's overall subject routes correctly, while a question about one unusual clause buried in it may never reach stage two at all — the clause is in the chunk pool, but its document was never selected.

Commonly confused with

A document summary index is not a summarisation feature, and the summaries it builds are not meant to be read by users. They exist as routing keys: their job is to be a good target for a document-level similarity match, which is a different objective from being a good summary for a human. It is also not the same as hierarchical or recursive summarisation such as RAPTOR, which builds a tree of summaries over clusters of chunks and can answer at several levels of abstraction. This index is deliberately two levels — document and chunk — which makes it cheaper to build and easier to reason about, at the cost of having no intermediate representation between a 300-token gist and a single chunk.

When to use it

Reach for it when:

For corpora of coherent whole documents, where document-level relevance is a meaningful signal.
On global questions about a document's content, which flat chunk ranking answers only by luck of fragment selection.
Where the document count is far smaller than the chunk count, since that ratio is what the first stage exploits.
When index-time cost is acceptable in exchange for per-query narrowing, as the $45 one-off above illustrates.

Reach for something else when:

For needle-in-a-haystack retrieval of specific clauses, which a 20-to-1 summary compression can route past entirely.
On heterogeneous documents covering many unrelated subjects, where one summary cannot represent the whole.
Over frequently changing corpora, since every document edit invalidates its summary and incurs regeneration.
Where multi-level abstraction is needed, which a two-level index does not provide and a summary tree does.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Document Summary Index — AI Glossary.

Frequently asked questions

What is Document Summary Index?