Logit Bias — AI Glossary

Reference details

Related terms

• Log Probabilities
• Sampling (LLM)
• Stop Sequence

How it works

1. The model produces a logit per vocabulary token for the next position. Logit bias is applied to that vector before any sampling transformation, adding a fixed value to the named token ids.
2. Because sampling probabilities come from a softmax over logits, adding a constant to one token's logit multiplies its odds against every other token by e raised to that constant. The effect is exactly multiplicative on odds, which is why the scale feels aggressive.
3. Bias is keyed by token id, not by string. The caller must tokenise with the model's own tokenizer, and a word that splits into several pieces needs each piece biased — which is why biasing a multi-token phrase rarely does what the caller intended.
4. A large negative bias suppresses a token to the point of practical exclusion rather than hard-banning it, since the softmax still assigns it a non-zero probability.
5. A large positive bias forces a token whenever it is a candidate at all, which is a blunt instrument: forcing a token at every position produces repetition, because the bias does not know which position you meant.
6. The bias is positional-blind and applies at every step, which is the central limitation. Constraining structure — matched braces, a schema, a grammar — needs a decoder that tracks state, not a constant added to a logit.

Example: What a bias of 5 actually does to the odds

Two candidate tokens have logits 2.0 and 1.0. Softmax turns the difference into odds: the first is favoured by e^(2.0 - 1.0) = e^1 = 2.72 to 1. Apply a bias of +5 to the second token and its logit becomes 6.0, so the odds become e^(2.0 - 6.0) = e^-4 = 0.018 to 1 in the first token's favour — the second is now favoured by about 54.6 to 1. The odds ratio moved by 2.72 / 0.018 = 148.4, which is exactly e^5, and that identity is the whole intuition: a bias of b multiplies the odds by e^b regardless of the starting logits. A bias of 1 is a 2.7x nudge, 5 is 148x, and 10 is e^10 = 22,026x. At -100 the factor is e^-100 ≈ 3.7 x 10^-44, which is why -100 is described as a ban. The practical consequence is that the useful range is narrow: anything above about 5 is not a preference, it is a decision, and treating the scale as linear is how callers accidentally force a token into every position.

Commonly confused with

Logit bias is not constrained decoding, and reaching for it to guarantee structure is the common mistake. A grammar- or schema-constrained decoder masks the entire set of tokens that would be invalid *at the current position*, recomputed as the output grows, so valid JSON is guaranteed. Logit bias is a constant applied at every position with no knowledge of what has been emitted, so it can make a token likely or unlikely but cannot make a structure correct. It is also not a content filter: suppressing tokens shifts the model onto synonyms and circumlocutions rather than removing the meaning.

When to use it

Reach for it when:

• Restricting a classification output to a small set of single-token labels, where biasing all other tokens down makes the output space effectively closed.
• Discouraging a specific known artefact — a boilerplate opener, a stray token the model over-produces — where a mild negative bias is enough.
• Forcing a binary choice, where both options are single tokens and the position is unambiguous.
• Experimenting with output preferences cheaply, before committing to a decoder change or a fine-tune.

Reach for something else when:

• Guaranteeing structured output. Use a schema-constrained decoder; bias has no notion of position or validity.
• Multi-token words or phrases, where biasing the first piece changes the tokenisation path and the result is rarely the phrase you wanted.
• Safety filtering, since suppressing tokens redirects the model to equivalent phrasings rather than refusing.
• Across models or tokenizers without re-deriving ids. Token ids are per-tokenizer, and a stale id map biases arbitrary unrelated tokens.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Logit Bias — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path