Model Collapse — AI Glossary

Reference details

Related terms

• Pretraining Data
• Data Augmentation
• AI Alignment

How it works

1. Each training generation samples from a model rather than from the world, and sampling under-represents the tails of a distribution — rare constructions, unusual facts, minority forms.
2. Training on that sample produces a model whose own distribution is narrower than its predecessor's, because it never saw what the sampling omitted.
3. Repeating the cycle compounds the narrowing geometrically rather than additively, which is why the failure is a collapse rather than a gradual decline.
4. Early generations lose variance while mean behaviour looks intact, so the degradation is invisible to accuracy metrics and visible only in diversity measures.
5. Later generations lose the mean too, as the surviving distribution concentrates on a shrinking set of modes and the model begins producing variations of a few patterns.
6. A fraction of real data mixed into every generation arrests it, because the tails are reintroduced from outside rather than resampled from a model that has already lost them.

Example: Why the loss compounds rather than accumulates

Suppose each generation of synthetic-only training retains 90% of the previous distribution's variance — a mild per-step loss that would be hard to notice in isolation. After two generations the retained variance is 0.9^2 = 0.81, after five 0.9^5 = 0.590, and after ten 0.9^10 = 0.349. So ten rounds of a 10% loss leaves 35% of the original diversity, not 100 - 10 x 10 = 0%, and not the 90% a single step suggests: the figure to reason about is the product, which is why the decline looks slow for several generations and then does not. At a harsher 80% retention it is 0.8^10 = 0.107, about a tenth, and at a gentle 95% it is 0.95^10 = 0.599. The mitigation works on the same arithmetic in reverse. Mixing real data in each round means the tails are resupplied externally rather than inherited from a model that has already lost them, so the per-generation retention stops being a multiplier below 1 — which is why the practical rule is a persistent human-data anchor in every generation rather than a larger synthetic set.

Commonly confused with

Model collapse is not overfitting and not catastrophic forgetting, and the distinction is what the model lost touch with. Overfitting is a model fitting its training set too closely and generalising worse, diagnosable by a train-test gap. Forgetting is a fine-tune degrading capabilities the base model had. Collapse is a distribution narrowing across generations of training on model output, so each generation is a reasonable fit to its own training data and the data itself has drifted from reality. No train-test gap reveals it, because the test set drifted too if it came from the same source.

When to use it

Reach for it when:

• As the standing risk in any synthetic-data pipeline, where the question is what fraction of real data anchors each round.
• When monitoring generated-data quality, where diversity measures catch it and accuracy measures do not.
• In data provenance decisions, since web-scraped text increasingly contains model output and the provenance is what distinguishes a real anchor from a synthetic one.
• As the argument for retaining human-authored corpora, whose value rises as the proportion of model-generated text rises.

Reach for something else when:

• As an argument against synthetic data generally, which is demonstrably useful in a single generation with real anchors.
• Diagnosed by accuracy, which stays intact while variance collapses — diversity is the measurement that sees it.
• Confused with overfitting or forgetting, which have different causes and different remedies.
• As a prediction about a specific model without knowing its data provenance, which is the variable that determines whether the mechanism applies at all.

Primary source

• Shumailov et al., The Curse of Recursion: Training on Generated Data Makes Models Forget (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Model Collapse — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path