Multi-Document Question Answering — AI Glossary

Reference details

Related terms

• Needle in a Haystack
• Open-Domain Question Answering
• RAG (Retrieval-Augmented Generation)

How it works

1. The question is decomposed, explicitly or implicitly, because the answer does not exist in any single passage.
2. Candidate passages are gathered from several documents — by retrieval, or by placing all of them in a long context window.
3. Entities are reconciled across sources, since the same referent appears under different names and a pronoun in one document may resolve only in another.
4. Facts are composed in order: an intermediate answer from one document conditions what must be found in the next, which is what makes it multi-hop rather than multi-source.
5. The answer is emitted with attribution per claim, because a synthesised answer that cannot be traced to its sources cannot be checked.

Example: Why the second hop is where retrieval fails

A question needs two facts: the author of a paper, then that author's institution. The first retrieval succeeds — the query terms match the paper's document. The second cannot, because the institution appears in a biography page sharing no terms with the original question; the query needed is the author's name, which was not known until the first hop returned. Single-shot retrieval over the original question therefore has no path to the answer however good the ranking, and this is the structural reason iterative retrieval exists.

Commonly confused with

Collapsed into RAG, which is a method rather than the task. RAG is one way to attempt multi-document QA, and long-context stuffing is another — put every document in the prompt and let attention do the work. They fail differently: retrieval misses the second hop when the query cannot be formed in advance, while stuffing degrades as relevant facts sit further apart in a long context. It is also not the same as single-document QA at larger scale: the defining difficulty is composing facts that no one passage contains, not finding a passage.

When to use it

Reach for it when:

• Evaluating whether a retrieval pipeline can answer questions requiring more than one source.
• Explaining why a system that answers single-fact lookups well fails on comparative or chained questions.
• Deciding between iterative retrieval and long-context stuffing for a corpus.

Reach for something else when:

• Questions answerable from one passage, however long the document.
• Summarisation over many documents, where nothing has to be composed to produce the output.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Multi-Document Question Answering — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path