Question Answering (QA) — AI Glossary

Reference details

Related terms

• Open-Domain Question Answering
• Multi-Document Question Answering
• RAG (Retrieval-Augmented Generation)

How it works

1. The question is first classified by what kind of answer it admits: a span present in some document, a synthesis across documents, or a fact the model must supply from its weights.
2. Extractive systems retrieve candidate passages and then predict a start and end offset within one of them, so the answer is a pointer into source text rather than new text.
3. Abstractive systems retrieve the same passages but generate the answer, which lets them merge several sources and rephrase, at the cost of the answer no longer existing anywhere verbatim.
4. Closed-book answering skips retrieval entirely and reads the answer out of parametric knowledge, which is fast and has no grounding to check against.
5. Production systems usually generate abstractively while carrying span-level attributions alongside, so each claim keeps a pointer back to the passage that supports it.
6. The final stage is abstention: a calibrated system declines when retrieval returned nothing relevant, which is the behaviour that distinguishes a QA system from a text generator pointed at questions.

Example: Why answer format drives the verification bill

Take a support QA system handling 10,000 questions a day, where 3% of answers get challenged by a user or flagged by sampling — that is 300 answers a day needing a human to establish whether they were right. If answers are abstractive prose with no attribution, verification means reading the sources afresh, call it four minutes each: 300 x 4 = 1,200 minutes, or 20 hours a day, which at an eight-hour shift is 20 / 8 = 2.5 full-time reviewers. Now carry spans. Verification becomes checking whether the cited span says what the answer claims, call it 30 seconds: 300 x 0.5 = 150 minutes, or 2.5 hours a day, which is 2.5 / 8 ≈ 0.31 of one reviewer. The two regimes are 20 hours against 2.5 hours of daily review for identical answer volume and identical answer quality, a factor of 8, and the difference is entirely a format decision made at design time. The generation cost moves the other way and by far less. Carrying attributions means emitting perhaps 80 extra output tokens per answer, so 10,000 x 80 = 800,000 tokens a day, and at $15 per million output that is $12 a day — against roughly 17.5 hours of recovered reviewer time.

Commonly confused with

Question answering is not retrieval, though the two are so often deployed together that the names get used interchangeably. Retrieval returns documents ranked by relevance and leaves the reading to the user; QA returns the answer and takes on the obligation to be right. That obligation is the whole difference, and it is why a QA system needs an abstention path while a search box does not — a search returning weak results is merely unhelpful, while a QA system answering from weak results is wrong. It is also distinct from reading comprehension as a benchmark task, where the passage containing the answer is supplied by the dataset; production QA has to find the passage first, and most production failures are retrieval failures wearing an answer's clothing.

When to use it

Reach for it when:

• When users arrive with specific questions rather than browsing intent, which is where returning an answer beats returning ten links.
• Over a corpus the user cannot reasonably read, since the value of synthesis rises with the document count.
• With span-level attribution carried through, which cuts the per-challenge verification cost by roughly eightfold in the worked case above.
• Where an abstention path exists, so the system can decline rather than answer from irrelevant retrieval.

Reach for something else when:

• For exploratory information needs, where a ranked list lets the user steer and a single answer forecloses it.
• Closed-book over facts that change, where parametric knowledge is stale and has no retrieval to correct it.
• Without attribution in any setting where answers will be challenged, since unattributed prose moves the cost onto human review.
• As a layer over retrieval that was never measured, given that most answer failures originate in the passages rather than the generation.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Question Answering (QA) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path