ModelRefs / Min-P Sampling — AI Glossary

Min-P Sampling — AI Glossary

A sampling method filtering tokens below a fraction of the top token's probability, adapting dynamically to distribution sharpness.

Overview

Min-p (Nguyen et al. 2023) sets a dynamic cutoff: keep any token whose probability ≥ p × p_max. When the model is confident (peaked distribution), the effective k is small; when uncertain, more tokens survive. Outperforms top-p on coherence-diversity trade-offs; adopted by llama.cpp, Ollama, and several frontier APIs.

Reference details

Topic	inference
Last reviewed	2026-06-24

How it works

The next-token distribution is computed as usual, and the highest probability in it is taken as a reference point.
A threshold is set as a fraction of that maximum, and every token below the threshold is removed from consideration before sampling.
The surviving set therefore adapts to the shape of the distribution: a peaked distribution produces a high threshold and few candidates, a flat one produces a low threshold and many.
That adaptation is the mechanism's point. The filter is relative to the model's own confidence at that position rather than to a fixed cumulative budget.
Remaining probabilities are renormalised and sampled, so temperature still applies on top and the two parameters interact.
The contrast with cumulative-probability sampling is that a fixed cumulative target must admit a tail of low-probability tokens whenever the head does not reach it, which is exactly where incoherent tokens enter.

Example: The case where the two methods diverge sharply

Take a confident position: one token at probability 0.9 and a hundred tokens at 0.001 each. A relative filter at 0.1 sets the threshold at 0.9 x 0.1 = 0.09, so only the 0.9 token survives — the candidate set is 1, which is what a confident model should produce. Now apply cumulative-probability sampling at 0.95. The head contributes 0.9, leaving 0.95 - 0.9 = 0.05 still to accumulate, which takes 0.05 / 0.001 = 50 of the low-probability tokens, giving a candidate set of 1 + 50 = 51. Fifty tokens the model assigned a thousand-to-one chance are now sampleable, and over a long generation one of them will be chosen. Raise the cumulative target to 0.99 and it admits 0.09 / 0.001 = 90 of them. The relative filter is unmoved by all of this, because it asks how each token compares to the best one rather than how much cumulative mass has been collected. In the flat case the two converge — with a hundred tokens at 0.01 each, a 0.1 relative threshold is 0.001 and keeps all of them, and a 0.95 cumulative target keeps 95 — which is why the divergence shows up specifically where the model is confident and a stray token is most damaging.

Commonly confused with

This is a truncation method, not a replacement for temperature, and they do different things in sequence. Truncation decides which tokens may be sampled at all; temperature reshapes the probabilities among whichever tokens remain. So a relative filter with a high temperature still samples only from the surviving candidates, and the two parameters have to be tuned together — tightening the filter and raising the temperature partly cancel. The comparison worth making is against cumulative-probability truncation, which it is designed to replace rather than supplement.

When to use it

Reach for it when:

Open-ended generation where both coherence and variety matter, which is the trade it was designed for.
Models and positions with sharply varying confidence, where a fixed cumulative budget behaves inconsistently.
Where occasional incoherent tokens are the observed failure, since a relative threshold is what excludes them at confident positions.
As a replacement for cumulative truncation rather than alongside it, since stacking both makes the effective cutoff hard to reason about.

Reach for something else when:

Deterministic tasks, where greedy decoding is the right choice and no truncation parameter is needed.
Where the runtime does not support it, which is still a real constraint across providers and frameworks.
Tuned independently of temperature, since the two interact and moving one changes what the other does.
Stacked with cumulative truncation, where two filters produce a cutoff neither parameter describes.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Min-P Sampling — AI Glossary.

Frequently asked questions

What is Min-P Sampling?