Uncertainty Quantification — AI Glossary

Reference details

Related terms

• Calibration
• Log Probabilities
• Abstention

How it works

1. Two distinct quantities are conflated under one word. Uncertainty from the data being genuinely ambiguous cannot be reduced by a better model; uncertainty from the model not knowing can be, and the response differs accordingly.
2. Sequence probability is the cheapest signal, derived from the token log-probabilities the model already produces, but it measures confidence in the wording rather than in the claim.
3. Verbalised confidence asks the model to state a number. It is trivially available and must be calibrated before it means anything, because a model trained to sound helpful tends to state high confidence.
4. Sampling-based agreement runs the same prompt several times and measures consistency. It is the most informative cheap method, because disagreement across samples is direct evidence that the model has not settled.
5. Conformal methods convert any confidence score into a set with a coverage guarantee, given a calibration set — which is the only approach that yields a statistical guarantee rather than a correlate.
6. All of them are judged by calibration rather than by accuracy: the question is whether a stated 80% is right 80% of the time, and that is measured by binning predictions and comparing claimed against observed.

Example: Measuring calibration rather than asserting it

Bin 300 answers by the confidence the model stated. Suppose the 100 answers claiming 90% confidence are right 70 times, the 100 claiming 70% are right 65 times, and the 100 claiming 50% are right 55 times. Each bin's calibration gap is the absolute difference between claimed and observed: |0.90 - 0.70| = 0.20, |0.70 - 0.65| = 0.05, and |0.50 - 0.55| = 0.05. The expected calibration error is the gap weighted by bin size: (100 x 0.20 + 100 x 0.05 + 100 x 0.05) / 300 = (20 + 5 + 5) / 300 = 30 / 300 = 0.10, or 10 points. Two things follow that a single accuracy figure would not show. First, the overall accuracy here is (70 + 65 + 55) / 300 = 63.3%, which says nothing about whether the confidence was usable. Second, the error is concentrated entirely in the high-confidence bin — 20 points of the 30 — which is the worst place for it, because high-confidence answers are the ones a downstream system acts on without review. A model that is overconfident only when it is confident is more dangerous than one that is uniformly 10 points off.

Commonly confused with

Confidence and calibration are different claims and only the second is checkable. A confidence score is the model's assertion about one answer; calibration is the empirical property that scores of a given level are right at that rate, measured over many answers against outcomes. So a model can produce finely-graded confidence scores that are entirely uninformative, and the only way to know is to bin and compare. The related error is treating sequence probability as answer confidence: a model can be highly confident about the wording of a claim it has no basis for, since the two are different quantities that happen to live on the same scale.

When to use it

Reach for it when:

• Abstention and routing decisions, where the whole purpose is identifying which answers not to trust.
• High-stakes deployments, where knowing when the model is unreliable matters as much as the answer.
• Sampling-based agreement as the default cheap method, since inconsistency across samples is direct evidence.
• With a calibration measurement, which is what turns a score into a usable signal.

Reach for something else when:

• Verbalised confidence taken at face value, which is uncalibrated by default and biased upward.
• Sequence probability read as confidence in the claim, which it does not measure.
• Without binning against outcomes, where a plausible-looking score may carry no information.
• Where the uncertainty is in the data rather than the model, since a better model does not reduce it and the right response is to say the question is ambiguous.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Uncertainty Quantification — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path