Modular RAG — AI Glossary

Reference details

Related terms

• Advanced RAG
• Agentic RAG
• Retrieval Pipeline

How it works

1. Each stage is given a declared input and output contract, which is what allows one implementation to be replaced by another without the callers changing.
2. Stages are registered and assembled by configuration rather than by code, so a pipeline becomes a described composition instead of a function calling the next function.
3. A router inspects the query and selects which path it takes, which is the module that makes several specialised pipelines coexist rather than one pipeline serving every case.
4. Where more than one retrieval path runs, a fusion module merges their results into a single ranking, since the paths return incomparable scores.
5. Every stage is instrumented separately, because the point of the decomposition is to locate which stage lost quality and that requires per-stage measurement.
6. The framing therefore specifies seams rather than components: nothing in it says which modules a pipeline should have, only that a pipeline should be made of replaceable ones.

Example: Composition, so a stage can be replaced — and what that saves

Retrieval quality is poor for one document type. In a modular pipeline you swap that type's retriever and leave everything else alone. In a monolithic one you change shared code and re-validate the whole system. The value is the seam, not any particular module. The saving is countable once the stages are named. Take a pipeline of six — query rewriting, retrieval, filtering, reranking, compression, generation. Changing one stage in a composed pipeline means evaluating that stage plus one integration run, so 2 evaluation cycles. Changing shared code in a monolithic one means re-validating all 6, because nothing establishes that the other five were untouched. That is a factor of 3 per change, and it recurs on every change rather than once — which is the argument for the seams and also the reason they are only worth building where changes are frequent.

Commonly confused with

Modular RAG is a framing, not an architecture you can install. It describes decomposing a pipeline into swappable stages — routing, memory, fusion, prediction. Nothing about the term specifies which modules, so it constrains a design far less than it appears to. The practical consequence is that the term cannot be a requirement: asking for a modular pipeline specifies no interface, no stage list and no contract, so two systems both described this way may share nothing beyond the claim.

When to use it

Reach for it when:

• Different content types genuinely need different handling
• You are running retrieval strategies in parallel and fusing them
• Stages need to be evaluated and replaced independently
• Where changes are frequent enough that the 3-to-1 re-validation saving recurs

Reach for something else when:

• One corpus, one query type — modularity is overhead with no payoff
• Before naive RAG works and is measured
• Module count is being treated as a proxy for sophistication
• As a specification, since the term fixes no interface and no stage list

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Modular RAG — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path