Top-P (Nucleus Sampling) — AI Glossary

Reference details

Related terms

• Temperature

How it works

1. At each step the model produces a probability for every token in the vocabulary.
2. Tokens are sorted by probability and accumulated until the running total reaches p, and everything after that point is discarded.
3. The surviving set is renormalised to sum to one and sampled from, so the cut changes which tokens are possible rather than reweighting all of them.
4. The set size therefore varies by position: a confident step may keep one token, an ambiguous one may keep hundreds.
5. Temperature is applied before the cut, so the two interact — raising temperature flattens the distribution and admits more tokens at the same p.

Example: Why the cut adapts and top-k does not

A confident step has probabilities 0.95, 0.03, 0.01, …: at p = 0.9 the cumulative total passes 0.9 on the first token, so exactly one survives and generation is effectively deterministic there. An ambiguous step with 50 tokens near 0.02 each needs 45 of them to reach 0.9, so 45 survive. Top-k = 10 would have kept 10 in both cases — truncating the ambiguous step's real options while admitting 9 implausible tokens at the confident one. The adaptivity is the whole point.

Commonly confused with

Top-p and top-k both restrict which tokens can be sampled, and differ in how the cut is decided: top-k always keeps a fixed number, top-p keeps as many as it takes to reach a probability mass, so it widens where the model is unsure and narrows where it is confident. Temperature is a separate control that reshapes the distribution before either cut. Changing more than one at a time makes the result uninterpretable.

When to use it

Reach for it when:

• Sampling should stay varied where the model is genuinely uncertain and tighten where it is not.
• Tuning creative output, where the tail should be trimmed without fixing a count.
• Avoiding low-probability tokens that cause derailment, without suppressing legitimate alternatives.

Reach for something else when:

• Deterministic output is required, where temperature 0 or greedy decoding is the control.
• Tuning it simultaneously with temperature and top-k, where three interacting knobs make the cause of a change unattributable.
• Structured output constrained by a schema, where the grammar already restricts the valid set.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Top-P (Nucleus Sampling) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path