Working Memory (Agents) — AI Glossary

Reference details

Related terms

• Memory Store
• Context Management
• Long-Term Memory (Agents)

How it works

1. Working memory is the context window as it stands during this step: system prompt, history, retrieved documents, tool results and any scratchpad.
2. It is reconstructed on every call rather than persisting — nothing carries over except what the caller resends.
3. Its contents compete for one finite budget, so a long history directly reduces how much retrieved evidence can be supplied.
4. Attention over it is uneven, so material in the middle is used less reliably than material at the edges.
5. Management is active: compress history, drop stale tool output, and re-place the most important material where it will be attended to.

Example: The budget is zero-sum

A 128,000-token window with a 2,000-token system prompt and 4,000 reserved for output leaves 122,000. A 60-turn history at 500 tokens each takes 30,000, leaving 92,000 for retrieval. Let the history reach 160 turns — 80,000 tokens — and retrieval is down to 42,000, less than half. No retrieval setting changed; the conversation consumed the budget. This is why summarising history is a retrieval-quality decision and not only a cost one.

Commonly confused with

Working memory is what is in the context window during this step; long-term memory is what survives the session. The practical difference is what each costs and how each fails: working memory is re-sent and re-billed on every call and disappears when the run ends, while long-term memory costs a retrieval and fails by returning the wrong entry. `memory` is the umbrella above both.

When to use it

Reach for it when:

• Reasoning about what an agent can actually see at a given step.
• Deciding what to compress or drop as a conversation or task lengthens.
• Explaining why retrieval quality degrades over a long session with no configuration change.

Reach for something else when:

• Describing what survives the session, which is long-term memory.
• Assuming it persists — it is rebuilt from what the caller sends on each call.
• Treating the window as free: everything in it is paid for on every turn.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Working Memory (Agents) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path