DSPy — AI Glossary

Reference details

Related terms

• Automatic Prompt Engineering (APE)
• Chain of Thought (CoT)
• Few-Shot Prompting

How it works

1. A task is declared as a signature — typed inputs and outputs — rather than written as a prompt string.
2. Modules compose those signatures into a program, so control flow lives in code and the prompt text does not exist yet.
3. A metric and a small labelled set define success, which is what makes the next step possible at all.
4. A compiler then searches over prompts and few-shot example selections, scoring candidates against the metric.
5. The compiled artefact is prompts plus selected demonstrations, so changing model requires recompiling rather than rewriting by hand.

Example: What recompiling buys when the model changes

A hand-tuned prompt is tuned for one model's quirks. Swap the model and the prompt is now tuned for the wrong thing, so someone re-tunes by hand and the quality after the swap depends on how much effort they spend. With a metric and a labelled set, the same program recompiles against the new model and the search does that work — the cost is the compile, and the result is comparable because both were optimised against the same metric. The prerequisite is the metric, which is also the thing most teams do not have.

Commonly confused with

Described as a prompting framework, which inverts its premise: the point is that you do not write prompts, you declare behaviour and compile it. Against orchestration frameworks the difference is what they manage — those compose calls, retrievers and tools with prompts you author, while this generates the prompts and leaves composition to your code. Its hard requirement is the metric: without a reliable score and a labelled set there is nothing to compile against, and it degrades into a more abstract way of writing the prompt by hand.

When to use it

Reach for it when:

• A reliable metric and a labelled set exist, so programs can be compiled rather than tuned.
• Models or providers change and prompts should be re-optimised rather than rewritten.
• Prompt text should be generated artefact rather than source to maintain.

Reach for something else when:

• No reliable metric exists, which removes the compiler's objective.
• One prompt for one task, where hand-writing is faster than setting up the program.
• The need is orchestration of tools and retrievers, which other frameworks target.

Primary source

• Khattab et al., DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to DSPy — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path