RAPTOR — AI Glossary

Reference details

Related terms

• Modular RAG
• Query Decomposition
• Advanced RAG

How it works

1. Leaf chunks are produced as in any retrieval pipeline, and that chunking decision still governs everything above it.
2. Those chunks are embedded and clustered by similarity, so semantically related passages are grouped regardless of where they sit in the document order.
3. Each cluster is summarised by a model, and those summaries are embedded as new nodes — a level of representation that did not exist in the source text.
4. The process recurses: summaries are clustered and summarised again, building a tree whose height is logarithmic in the leaf count.
5. Retrieval queries every level at once rather than descending the tree, so a single search can return both a specific passage and a summary that frames it.
6. That is the mechanism's point: a question needing a global synthesis matches a high-level summary, and one needing a specific fact matches a leaf, without the retriever knowing which kind of question it was given.

Example: What the tree costs to build and store

Start with 10,000 leaf chunks and cluster at roughly 10 to 1 per level. The first summary level holds 10,000 / 10 = 1,000 nodes, the second 100, the third 10, and the fourth 1 — so the tree is log base 10 of 10,000 = 4 levels above the leaves. Total nodes are 10,000 + 1,000 + 100 + 10 + 1 = 11,111, which is 1,111 / 10,000 = 11.1% more vectors to store and search than flat retrieval. That is the cheap part. The expensive part is construction: 1,111 summary nodes each require a model call over the text of its cluster, and at roughly 5,000 input tokens per cluster that is 1,111 x 5,000 = 5,555,000 input tokens, or at 3 units per million, about 5.56 x 3 = 16.7 units — plus output. So the index costs a one-time model run rather than just an embedding pass, and it must be rebuilt when the corpus changes. The 11% storage overhead is not the number that decides whether to use this; the rebuild cost on a changing corpus is.

Commonly confused with

This is a hierarchical index, not a reranking or query-expansion step, and the difference is where the new information comes from. Reranking reorders candidates the retriever already found; query expansion issues more searches over the same corpus. A summary tree adds representations that are not in the corpus at all — a cluster-level synthesis that no single chunk contains — which is why it helps on questions whose answer is distributed across many passages and no individual chunk answers. The cost side of that distinction is that the added representations are model-generated, so they inherit the summariser's errors.

When to use it

Reach for it when:

• Questions requiring synthesis across a document or corpus, where no single chunk contains the answer.
• Mixed query workloads combining specific lookups with global questions, which multi-level retrieval serves with one index.
• Stable corpora, where a one-time construction cost amortises over many queries.
• Long documents with thematic structure, where clustering finds groupings that document order does not.

Reach for something else when:

• Frequently changing corpora, where the rebuild cost rather than the storage overhead is the binding constraint.
• Purely extractive lookup workloads, where flat retrieval finds the chunk and the summary levels are unused.
• Where summariser errors are unacceptable, since the added nodes are generated text and are retrieved as if they were source.
• Before flat retrieval has been evaluated, since this adds a construction pipeline on top of a chunking decision that may be the actual problem.

Primary source

• Sarthi et al., RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to RAPTOR — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path