Extended Thinking — AI Glossary

Reference details

Related terms

• Chain of Thought (CoT)
• Reasoning (LLM)
• Tree of Thoughts (ToT)

How it works

1. The model is given a budget of tokens to emit before its visible answer begins, and those tokens are generated by the same forward pass machinery as any output.
2. Training rather than prompting is what makes them useful: reinforcement learning on problems with checkable answers rewards traces that lead to correct final answers.
3. Because the trace is autoregressive, each step conditions on the previous ones, so the model can commit to an approach, notice it fails, and abandon it within a single response.
4. The trace is usually withheld from the caller, which keeps it free of the presentational pressure that shapes user-visible text and lets it contain discarded work.
5. Thinking tokens are billed at the output rate and occupy the context window, so the budget competes directly with the material the task needs.
6. A budget set too low truncates the reasoning mid-approach, which is worse than no trace at all, because the answer is then conditioned on an abandoned line of work.

Example: Most of the output bill is text nobody reads

Take a task that produces 1,200 visible output tokens after 8,000 thinking tokens. Both are billed at the output rate, so at $15 per million the visible answer costs 0.0012 x $15 = $0.018 and the trace costs 0.008 x $15 = $0.120, for $0.138 in total — of which the trace is 0.120 / 0.138 ≈ 87.0%. Nearly nine tenths of the output charge is text the user never sees. Latency moves with it. At 60 output tokens per second, 9,200 tokens take about 153.3 seconds against 20.0 seconds for the visible answer alone, a factor of 9,200 / 1,200 ≈ 7.67. Whether that is worth paying depends entirely on the failure rate of the alternative, which makes it a calculable decision rather than a preference. Suppose the direct answer is acceptable 55% of the time and the thinking answer 88%. Cost per acceptable answer is 0.018 / 0.55 ≈ $0.0327 against 0.138 / 0.88 ≈ $0.1568, so thinking costs about 4.79 times more per success while adding 33 points of accuracy. On a task where a wrong answer is cheap to discard, the direct path wins on this arithmetic; on one where a wrong answer reaches a customer, 4.79 times the unit cost is rarely the expensive side of the ledger.

Commonly confused with

Extended thinking is not chain-of-thought prompting, and the difference is where the behaviour comes from. Chain-of-thought is a prompting technique that asks a model to show its steps, and the steps are part of the visible answer; extended thinking is a trained mode in which a separate budget of hidden tokens is spent before the answer begins, reinforced on whether the final answer was right. One can be applied to any model by changing the prompt, the other cannot be prompted into a model that was not trained for it. The second confusion is treating the hidden trace as an explanation. It is a work surface, containing approaches the model tried and rejected, and reading it as the model's reasoning about its final answer attributes to it a coherence it does not claim.

When to use it

Reach for it when:

• On problems with many interacting steps, where the ability to abandon an approach mid-response is the mechanism that pays.
• Where a wrong answer is expensive downstream, which is what justified roughly 4.79 times the unit cost above.
• With a budget set generously enough to complete a line of reasoning, since a truncated trace is worse than none.
• When latency is not user-facing, given the worked case ran about 7.67 times longer.

Reach for something else when:

• On lookup, extraction, formatting or classification, where there is no multi-step structure for the trace to exploit.
• In interactive paths with a latency budget, where 153.3 seconds against 20.0 is the trade being made.
• At high volume on cheap tasks, since the trace was about 87.0% of the output bill.
• Read as an explanation of the answer, which is not what a scratchpad containing discarded work is.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Extended Thinking — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path