Query Decomposition — AI Glossary

Reference details

Related terms

• Query Rewriting
• Multi-Document Question Answering
• Advanced RAG

How it works

1. The incoming question is analysed for whether it contains more than one retrievable fact, since a single-fact question needs no decomposition.
2. It is split into atomic sub-questions, each answerable from one place in the corpus.
3. Independent sub-questions are retrieved in parallel; dependent ones must wait, because the query for the second is only formable once the first has returned.
4. Each sub-answer is kept with its own evidence, which is what makes the final answer attributable claim by claim.
5. The sub-answers are synthesised, and a missing one has to be surfaced rather than smoothed over — an incomplete synthesis presented as complete is the failure mode.

Example: Why dependency decides the latency

A comparison question splits into two independent sub-questions at 1.5s each plus a 2s synthesis: run in parallel that is 1.5 + 2 = 3.5s. A chained question — find the acquirer, then find that company's founder — cannot parallelise: 1.5 + 1.5 + 2 = 5s, and the second retrieval was impossible before the first returned. Same number of sub-questions, different critical path. The dependency structure, not the count, is what sets the cost.

Commonly confused with

For questions that need more than one retrieval to answer — comparisons, or anything requiring a fact found in one place and a fact found in another. Rewriting and expansion both still issue a search for a single question; decomposition accepts that no single search can succeed, so it is the right tool exactly when the query is multi-hop and the wrong one when it is merely vague.

When to use it

Reach for it when:

• Questions compare two things, or need a fact from one place and a fact from another.
• A single retrieval reliably returns evidence for one half of the question only.
• Per-claim attribution is required in the final answer.

Reach for something else when:

• Single-fact lookups, where decomposition adds a planning call for nothing.
• Latency cannot absorb sequential retrieval for dependent sub-questions.
• Synthesis cannot signal a missing sub-answer, in which case partial evidence reads as complete.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Query Decomposition — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path