ROUGE — AI Glossary

Reference details

Related terms

• BLEU
• BERTScore
• Exact Match (EM)

How it works

1. Both candidate and reference are tokenised, and the metric counts overlapping units rather than judging meaning.
2. ROUGE-N counts matching n-grams and divides by the n-grams in the reference, which makes it recall-oriented: it asks how much of the reference was covered.
3. ROUGE-L finds the longest common subsequence, so word order is rewarded without requiring contiguity.
4. With several references, the score is taken against the best-matching one, since no single reference is the only acceptable summary.
5. Scores are averaged over the corpus. A per-example figure is close to meaningless; the metric is only stable in aggregate.

Example: Where lexical overlap stops measuring quality

Reference: 'the cat sat on the mat' — six unigrams. Candidate: 'the cat sat on a mat' matches five of them, so unigram recall is 5/6 ≈ 0.83. Now a candidate that is a perfect paraphrase, 'the feline rested upon the rug', matches 'the' twice and nothing else: recall 2/6 ≈ 0.33. The second summary is arguably better and scores less than half as well. That gap is not a flaw in the implementation — it is what counting n-grams measures, and it is why ROUGE is weak on abstractive summarisation and still useful on extractive.

Commonly confused with

Paired with BLEU as though interchangeable, and they are oriented oppositely. BLEU is precision-oriented with a brevity penalty, built for translation where output length is constrained; ROUGE is recall-oriented, built for summarisation where the question is coverage. Using BLEU for summarisation rewards terseness and ROUGE for translation rewards padding. Both are lexical, which is the deeper point: neither can credit a correct paraphrase, which is why embedding-based scores and LLM judges displaced them for abstractive work while they remain reasonable for extractive.

When to use it

Reach for it when:

• Extractive summarisation, where good output does reuse the source's wording.
• Regression testing a summariser cheaply and deterministically across builds.
• Reporting alongside a semantic metric, where the pair is more informative than either.

Reach for something else when:

• Abstractive summarisation, where a correct paraphrase is scored as a miss.
• Open-ended generation with no reference text to compare against.
• As a single quality gate, since it cannot distinguish fluent nonsense that reuses the right words.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to ROUGE — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path