Repetition Penalty — AI Glossary

Reference details

Related terms

• Sampling (LLM)
• Greedy Decoding
• Temperature

How it works

1. After the model produces logits for the next token, the decoder scans the tokens already present in the context and marks their vocabulary positions.
2. Each marked logit is divided by the penalty factor before the softmax, so the adjustment happens in log space rather than on probabilities.
3. The softmax then renormalises over the whole vocabulary, which means penalising one token raises every other token's probability, not only the intended alternative.
4. Frequency penalty differs by scaling the adjustment with the number of prior occurrences, so a token used five times is suppressed more than one used once.
5. Presence penalty applies a fixed subtraction on any occurrence at all, which makes it a switch on novelty rather than a gradient against repetition.
6. The division form has an asymmetry worth knowing: dividing a negative logit by a factor above one moves it toward zero, so a naive implementation rewards tokens the model disfavoured.

Example: What a penalty of 1.2 does to the distribution

Take three candidate tokens with logits 4.0, 3.5 and 3.0. Exponentiating gives 54.598, 33.115 and 20.086, summing to 107.799, so the probabilities are 0.506, 0.307 and 0.186 — the first token is the clear favourite. Now suppose the first token already appears in the context and a repetition penalty of 1.2 applies. Its logit becomes 4.0 / 1.2 ≈ 3.333, which exponentiates to 28.032, and the new sum is 28.032 + 33.115 + 20.086 = 81.233. The probabilities are now 0.345, 0.408 and 0.247: the favourite has fallen from 0.506 to 0.345 and the second token has become the most likely at 0.408. A moderate penalty reordered the top two without eliminating anything. Push it to 2.0 and the behaviour changes character. The first logit becomes 2.0, exponentiating to 7.389, the sum is 60.590, and its probability is 7.389 / 60.590 ≈ 0.122 — the token that was most likely at 0.506 is now the least likely of the three. That is how an aggressive penalty breaks fluent text: the tokens a language repeats most are its function words and the subject's own name, so suppressing repetition suppresses exactly the vocabulary coherent prose depends on. The negative-logit asymmetry compounds it. A logit of -2.0 divided by 1.2 becomes about -1.667, and since exp(-2.0) ≈ 0.135 while exp(-1.667) ≈ 0.189, the penalty has made a disfavoured token roughly 1.4 times more likely instead of less.

Commonly confused with

Repetition penalty is not temperature, and they fail in opposite directions. Temperature rescales every logit uniformly, changing how sharply the distribution concentrates without preferring any particular token; repetition penalty is targeted, altering only the logits of tokens already seen. Raising temperature to escape a repetition loop therefore makes the whole output less predictable rather than specifically less repetitive. The second confusion is among the three penalty parameters, which providers expose under similar names. Repetition penalty divides logits multiplicatively, frequency penalty subtracts in proportion to count, and presence penalty subtracts a constant for any occurrence — so the same numeric value means three different things, and a value tuned on one does not transfer to another.

When to use it

Reach for it when:

• To break degenerate loops in greedy or low-temperature decoding, which is the failure the parameter exists for.
• At mild settings in the 1.1 to 1.2 range, which weaken the leading token rather than removing it — from 0.506 to 0.416 at 1.1, and to 0.345 at 1.2 where the second token takes the lead.
• On open-ended generation, where some lexical variety is a quality the output is judged on.
• As a targeted fix when the diagnosis is specifically repetition, rather than general output instability.

Reach for something else when:

• On structured output, where valid JSON requires repeating keys, braces and field names by construction.
• At high values, since a penalty of 2.0 drove the leading token from 0.506 to 0.122 in the case above.
• On code generation, where identifiers and keywords must recur and suppressing them produces syntax errors.
• As a substitute for temperature or top-p, which control concentration rather than token-specific suppression.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Repetition Penalty — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path