Multi-Query Retrieval — AI Glossary

Reference details

Related terms

• Query Expansion
• Fusion Retrieval
• Dense Retrieval

How it works

1. A model rewrites the user's question into several variants, each phrased to surface a different facet — a synonym set, a decomposed sub-question, a hypothetical answer whose wording may match the documents better than the question does.
2. Each variant runs as an independent retrieval, which is the point: the variants fail differently, so a document missed by one may be found by another.
3. Results are merged and deduplicated by document identity. The merge strategy matters, since naive concatenation lets one variant dominate and reciprocal-rank fusion weights agreement across variants.
4. The retrievals are independent and so should run concurrently, making wall-clock latency the slowest variant rather than the sum.
5. The gain comes entirely from the variants being decorrelated. Rewrites that are near-paraphrases retrieve the same documents and add cost without recall.
6. Cost scales with the variant count on the retrieval side plus one extra model call to generate them, and a larger merged candidate set then pushes more work onto whatever reranks it.

Example: The recall ceiling, and why the real gain is smaller

Suppose a single query retrieves the relevant document 60% of the time. If three variants were fully independent, the chance that all three miss is 0.4^3 = 0.064, so union recall would be 1 - 0.064 = 0.936 — a jump from 60% to 94%. That is the ceiling, and it is not achievable, because variants generated from one question are correlated: they share vocabulary, intent and the same embedding model's blind spots. If the variants are highly correlated, union recall approaches the 60% of a single query and the extra two retrievals bought nothing. Realistic gains land between, and the figure that predicts where is how different the retrieved sets actually are — which is directly measurable. Compute the overlap: if three variants return sets sharing 80% of their documents, only 20% of each is new and the method is close to its useless end; at 40% overlap it is working. That measurement costs nothing and is the honest way to decide whether to keep paying 3x retrieval, instead of assuming the independence that produced the 94%.

Commonly confused with

Multi-query retrieval is not query decomposition, though an implementation may do both. Multi-query generates alternative phrasings of the same question and unions the results, so it raises recall on one information need. Decomposition splits a question into different sub-questions whose answers must be combined, so it addresses multi-hop needs and the results are not interchangeable — each sub-answer is required rather than redundant. Merging decomposed results by deduplication, as if they were variants, discards the structure the decomposition created.

When to use it

Reach for it when:

• Recall-critical retrieval where missing a document is expensive and extra latency is acceptable.
• Ambiguous or underspecified queries, where several readings are plausible and covering them is the goal.
• Vocabulary mismatch between how users ask and how documents are written, which rewriting addresses directly.
• With measured overlap between variant result sets, so the decision to keep paying for it rests on evidence.

Reach for something else when:

• Latency-sensitive paths, where the extra generation call plus wider retrieval is felt.
• Precise, well-specified queries, where one retrieval already finds the document and variants add noise.
• When variants come back as near-paraphrases, retrieving the same set — measure the overlap before assuming a gain.
• Multi-hop questions needing composition, where decomposition rather than variation is the right tool.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Multi-Query Retrieval — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path