Agentic RAG — AI Glossary

Reference details

Related terms

• Modular RAG
• Self-RAG
• AI Agent

How it works

1. Retrieval is exposed as a tool rather than hardwired as a first step, so the model decides whether the question needs the corpus at all.
2. The model writes the query, which lets it rephrase the user's wording into terms the index actually contains.
3. Returned passages are assessed before use, and a bad result can be rejected rather than passed to generation as though it were relevant.
4. A follow-up query can be issued using what the first return revealed, which is what makes a second hop possible at all.
5. The loop ends on the model's own judgement that context is sufficient, so it needs a step cap and a cost cap imposed from outside.

Example: The cost of putting the decision in the model

A fixed pipeline is one retrieval and one generation: two units of work, every time. An agentic loop that averages 2.5 retrievals plus assessment and a final generation is nearer six. That is roughly 3x the cost and latency per question. It buys the questions a single retrieval cannot answer — the second hop, the rephrase, the decision not to retrieve. If most traffic is single-fact lookups, the fixed pipeline is the better engineering; the arithmetic only favours the loop when the hard questions are common enough to pay for the easy ones.

Commonly confused with

The retrieval decision moves into the model: whether to retrieve at all, what to search for, and whether to search again. That is the difference from a fixed pipeline, which always retrieves exactly once. It overlaps with Self-RAG and Corrective RAG, which are specific published mechanisms for parts of the same idea, whereas this names the general pattern.

When to use it

Reach for it when:

• Questions need more than one retrieval, where the second query depends on the first result.
• User phrasing differs from corpus vocabulary and the query must be rewritten to match.
• Some questions need no retrieval, and retrieving anyway injects irrelevant context.

Reach for something else when:

• Traffic is dominated by single-fact lookups a fixed pipeline answers at a fraction of the cost.
• Latency budgets cannot absorb several sequential retrievals.
• No step or cost cap is in place, leaving the loop to decide its own spend.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Agentic RAG — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path