Open-Domain Question Answering — AI Glossary

Reference details

Related terms

• Retrieval Pipeline
• RAG (Retrieval-Augmented Generation)
• Multi-Document Question Answering

How it works

1. The question arrives with no attached passage, so the system must first decide where the answer could come from: the model's parameters, a retrieval corpus, or both.
2. In the retrieval route a retriever scores the corpus against the question and returns the top k passages. Recall at k is the ceiling on everything downstream — a passage the retriever misses cannot be read by any reader.
3. A reader then extracts or generates the answer from those passages. The classic split is extractive, selecting a span, versus generative, where several passages are fused and the answer is written rather than copied.
4. The parametric route skips retrieval and relies on facts encoded in weights during pretraining. It is fast and has no corpus to maintain, and it cannot cite, cannot be updated, and degrades on anything after the training cutoff.
5. Scoring is exact match plus token F1 against short reference answers, which means an answer that is correct but phrased differently can score zero — the metric, not the model, is often what caps a reported number.
6. The practical system is hybrid: retrieve, generate with citations, and abstain when retrieval returns nothing relevant. Abstention is the part most implementations omit and the part that makes the output trustworthy.

Example: Why recall at k is the number to fix first

A reader converts a correct passage into a correct answer 80% of the time. If the retriever's recall at 5 is 70%, end-to-end accuracy is at most 0.70 x 0.80 = 0.56. Improving the reader to a perfect 100% raises the ceiling only to 0.70 — a 14-point gain. Improving recall at 5 from 70% to 90% while leaving the reader at 80% gives 0.90 x 0.80 = 0.72, a 16-point gain, and it compounds with any later reader work. This is why retrieval evaluation comes before generation evaluation, and why 'the model hallucinated' is frequently a retrieval report rather than a model one: at recall 70%, roughly 3 questions in 10 reach the reader with no supporting passage at all, and a reader that answers anyway is doing the only thing it was asked to do.

Commonly confused with

Open-domain QA is not reading comprehension, and the difference is the retrieval step. A reading-comprehension benchmark hands the model the passage containing the answer and measures extraction; open-domain hands it only the question. A system that scores well on the former can fail entirely on the latter, because the hard part — finding the right passage among millions — has been removed. It is also not the same as RAG: RAG is an architecture, open-domain QA is a task, and the task can be attempted parametrically with no retrieval at all.

When to use it

Reach for it when:

• Users ask factual questions without supplying a source, which is the shape of most search and assistant traffic.
• The answer must be current or verifiable, where retrieval supplies both freshness and a citation.
• A corpus exists that genuinely contains the answers, so recall is achievable rather than aspirational.
• Benchmarking retrieval and generation jointly, since open-domain accuracy is the only number that reflects the whole pipeline.

Reach for something else when:

• The answer requires synthesis across many documents or a judgement call. Short-answer QA metrics do not measure that, and will report failure on good output.
• The corpus does not contain the answer. Then the only correct behaviour is abstention, and an accuracy metric will punish it.
• Questions are ambiguous or presuppose something false, where a single reference answer is not well defined.
• Comparing systems whose retrieval corpora differ. The corpus is a larger factor than the model, so the comparison measures the index.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Open-Domain Question Answering — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path