Conversation History — AI Glossary

Reference details

Related terms

• Multi-Turn Conversation
• Context Window
• Context Management

How it works

1. The endpoint holds no session state, so the client stores the transcript and resends it on every call.
2. History is a list of role-tagged turns, and the model sees it as tokens rather than as remembered state — which is why an early instruction can fade rather than persist.
3. Input grows each turn, so cost and prefill latency rise across a conversation even when the newest message is short.
4. As the transcript approaches the window, something is dropped or compressed: truncation loses the oldest turns, summarisation loses detail.
5. Retrieved context competes for the same budget, so a long history directly reduces how much evidence a RAG step can supply.

Example: History and retrieval compete for one budget

A 32,000-token window, a 1,000-token system prompt and a 2,000-token reserve for the answer leaves 29,000 for history plus retrieved context. Twenty turns at 400 tokens each is 8,000, leaving 21,000 for retrieval. At sixty turns history is 24,000 and retrieval is squeezed to 5,000 — a quarter of what it had. Nothing about the retrieval configuration changed; the conversation consumed its budget, which is why summarisation is a retrieval-quality decision as much as a cost one.

Commonly confused with

The history is the data you resend on every request; multi-turn is the property of the interaction that results. Keeping them apart is useful because the problems are different: history is a cost and truncation problem, while multi-turn behaviour is a quality problem — instruction memory, coherence, handling a topic change. Context management is the set of strategies that connect the two.

When to use it

Reach for it when:

• Any multi-turn feature, where the resend policy must be an explicit decision.
• Deciding between truncation and summarisation before the window is actually hit.
• Diagnosing why retrieval quality degrades as a conversation lengthens.

Reach for something else when:

• Single-shot tasks with no prior turns.
• Assuming the provider keeps state — almost none do for completion endpoints.
• As a description of the interaction, which is multi-turn; this is the payload.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Conversation History — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path