Retry Logic — AI Glossary

Reference details

Related terms

• Tool Result
• Retry with Exponential Backoff
• Stopping Condition

How it works

1. The failure is classified first, because the right response differs: a malformed argument is the model's error, a timeout is the service's, and a validation rejection is the schema's.
2. For a model error, the error message is returned as a tool result so the model can see what was wrong and correct its own arguments.
3. For a transient service error, the same call is reissued with exponential backoff and jitter, with no model involvement at all.
4. A retry budget caps attempts per call and per run, so a self-correcting loop cannot spend unboundedly on one step.
5. Repeated identical failures must break the loop rather than retrying, since a model that produced the same wrong argument twice will usually produce it again.

Example: Why the two failure classes need different handling

A malformed-JSON tool call retried verbatim with backoff fails identically every time — the input was wrong, not the moment. Returning the parse error to the model lets it fix the arguments, and a single corrective turn often succeeds. Conversely a 503 handed back to the model wastes a reasoning turn on something it cannot fix. Misrouting either way produces a loop that burns tokens and never converges, which is the common shape of an agent that hangs.

Commonly confused with

Easily conflated with SDK-level retries, which handle a different class. A client library retries transport failures — timeouts, 429s, 5xx — and the model never learns it happened. This is agent-level: the tool returned a semantic error, and the fix is for the model to change what it asked for. The two compose, and conflating them produces the two characteristic bugs: backing off on a malformed argument that will never succeed, or asking a model to reason about a network error it cannot influence.

When to use it

Reach for it when:

• Agents call tools whose arguments the model constructs and can therefore get wrong.
• Tool errors carry information a model could act on, such as validation messages.
• A retry budget exists, so self-correction cannot run unbounded.

Reach for something else when:

• The failure is transport-level, where client backoff handles it without a model turn.
• The same failure has already recurred identically, which signals a loop rather than a retryable error.
• No budget caps attempts, letting one step consume the run.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Retry Logic — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path