What Is RAG? Retrieval-Augmented Generation Explained

Article

Plain-English definition

Retrieval-Augmented Generation (RAG) is a way of answering questions with a language model in two steps: first retrieve the most relevant pieces of text from a knowledge source (documents, wikis, tickets, manuals), then generate an answer using both the question and that retrieved text. The model reads the evidence you hand it instead of relying only on what it memorized during training.

The approach was formalized in a 2020 research paper from Facebook AI Research and has since become the default architecture for connecting language models to private knowledge.

Why it matters

Language models are trained once on a snapshot of text. Anything that happened after training, anything private to your organization, and anything too niche to be well represented in training data is invisible to the model. Yet these are exactly the questions businesses need answered: our policies, our product docs, this week's data.

RAG matters because it solves this without retraining. Updating the knowledge means updating documents, not the model. It also produces answers that can cite their sources, which makes outputs checkable, a property pure generation cannot offer.

How it works

A typical RAG pipeline has four stages:

1. Indexing. Documents are split into chunks, converted into embeddings (numeric representations of meaning), and stored in a searchable index, often a vector database. 2. Retrieval. When a question arrives, it is embedded the same way, and the index returns the chunks whose meaning is closest to the question. 3. Augmentation. The retrieved chunks are placed into the model's prompt, inside its context window -- alongside the question and instructions. 4. Generation. The model writes an answer grounded in the supplied chunks, ideally citing which ones it used.

A simple example

Imagine a support assistant for an airline. A customer asks: "Can I get a refund if my flight was delayed four hours?" Without RAG, the model answers from general training data, plausibly, but not from this airline's policy. With RAG, the system first searches the airline's policy documents, finds the section on delay compensation, and passes that exact text to the model. The answer now quotes the real policy, and the customer can be shown the source paragraph.

Common misunderstandings

- "RAG teaches the model new things." It doesn't. The model is unchanged; it just reads what you retrieved, the way you can read a reference sheet without memorizing it. - "RAG eliminates hallucination." It reduces it. The model can still misread evidence or answer past it. See LLM hallucination. - "Bigger context windows make RAG obsolete." Long context helps, but stuffing everything into the prompt is slower, costlier, and often less accurate than retrieving the right few passages. - "RAG is one thing." It's a family of designs -- chunking strategy, search type, reranking, and citation handling all vary and materially change quality.

Where RAG appears in real systems

Customer-support assistants grounded in help centers; enterprise search over wikis and drives; coding assistants that retrieve from your repository; legal and medical research tools that must cite sources; chatbots over product documentation. On ModelRefs, RAG-related implementation blueprints live in the workflow catalog, and model choice for RAG has its own decision guide.

Limitations and caveats

- Retrieval is the weakest link. If search returns the wrong chunks, the model answers fluently from wrong evidence. - Stale indexes. If documents change but the index isn't refreshed, answers cite outdated truth. - Chunking hurts context. Splitting documents can separate a rule from its exceptions. - Citations can drift. Models sometimes cite a retrieved source that doesn't actually support the sentence. Evaluation is not optional.

Next steps

Learn the building blocks: embeddings and vector databases. Ready to build? Start with the hands-on RAG tutorial, then the enterprise RAG workflow guide.

Sources and further reading

- Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (arXiv, 2020) -- the paper that introduced RAG. - OpenAI, Embeddings guide -- the retrieval representation RAG systems typically use.