ModelRefs / RAG (Retrieval-Augmented Generation) — AI Glossary
RAG (Retrieval-Augmented Generation) — AI Glossary
An architecture that retrieves relevant documents from an external store and injects them into the LLM prompt at inference time.
Overview
RAG separates knowledge (in a vector or keyword index) from reasoning (in the LLM). It enables citation, freshness, and per-tenant data without retraining. Typical pipeline: chunk → embed → store → retrieve → rerank → generate.
Reference details
| Topic | rag |
|---|---|
| Also known as | Retrieval Augmented Generation, retrieval-augmented generation |
| Last reviewed | 2026-06-24 |
Related terms
How it works
- Index time and query time are separate pipelines that share only the embedding model, and most of the engineering effort sits in the first while most of the attention goes to the second.
- At index time documents are split into chunks sized to the retrieval unit, each chunk is embedded, and the vectors are stored with metadata and a pointer back to the source.
- At query time the question is embedded by the same model, nearest chunks are retrieved, and a reranker usually reorders them because vector proximity ranks worse than a model reading the pair.
- The surviving chunks are assembled into the prompt inside an explicit frame telling the model to answer from the supplied material and to say so when it cannot.
- That instruction is load-bearing: without it the model has no way to distinguish context it should defer to from context it may ignore, and will fall back on its weights.
- Generation then produces the answer with pointers to the chunks that supported it, which is what makes the output checkable rather than merely plausible.
Example: Grounding an answer that changes weekly
“What is our current refund window?” Asked cold, the model answers from training data — a plausible 30 days that may be wrong. With RAG, the policy document is retrieved and injected, so the answer reflects the version in the index and can cite it. Change the policy, reindex, and the answer changes with no retraining.
Commonly confused with
RAG is not a fix for hallucination — it changes what the model hallucinates about. If retrieval returns nothing relevant, a model that has not been told to abstain will still answer, now with misplaced confidence because context was present. Groundedness has to be measured, not assumed.
When to use it
Reach for it when:
- The knowledge changes faster than you would retrain
- Answers must cite a source
- Data is per-tenant or access-controlled and cannot go into weights
Reach for something else when:
- You need a change in behaviour, tone or format — that is fine-tuning
- The whole corpus fits comfortably in context and stays small
- Retrieval quality is poor: bad context makes answers worse than no context
Referenced by
This term is used by the following ModelRefs references:
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to RAG (Retrieval-Augmented Generation) — AI Glossary.
Frequently asked questions
What is RAG (Retrieval-Augmented Generation)?
An architecture that retrieves relevant documents from an external store and injects them into the LLM prompt at inference time.
Is RAG (Retrieval-Augmented Generation) the same as Retrieval Augmented Generation?
Yes — Retrieval Augmented Generation, retrieval-augmented generation are common aliases for RAG (Retrieval-Augmented Generation).
What concepts are related to RAG (Retrieval-Augmented Generation)?
Closely related concepts include embedding, vector database, semantic search, grounding.