ModelRefs / Data Augmentation — AI Glossary

Data Augmentation — AI Glossary

Techniques generating additional training examples by transforming or synthesizing from existing data to improve coverage and robustness.

Overview

For text: back-translation, paraphrasing, instruction rewriting via stronger models. For multimodal: image cropping, rotation, color jitter. For code: variable renaming, dead-code injection. Synthetic data generation (GPT-4 generating diverse instructions) is the dominant LLM augmentation strategy; quality control is critical to avoid noise.

Reference details

Topic	training
Last reviewed	2026-06-24

How it works

A transformation is chosen that preserves the label while changing the surface form, which is the entire design constraint: a transformation that can flip the label manufactures noise rather than coverage.
For text the common operators are back-translation, paraphrasing with a stronger model, instruction rewriting, and controlled perturbation of entities or numbers. For code, semantics-preserving edits such as renaming variables or reordering independent statements.
Generated candidates are filtered before use. Deduplication against the original and against each other, a quality score or model-based judgement, and a label-consistency check are what separate augmentation from dilution.
The mixing ratio is set deliberately. Augmented data is correlated with its source, so n augmented copies of an example are worth far less than n independent examples and can bias the model toward the generator's style.
Evaluation is held clean. Augmenting the training set while the test set stays untouched is what makes the measured gain meaningful; augmenting both inflates the score without improving anything.
Synthetic generation from a stronger model is the dominant modern form, and it inherits that model's distribution — including its errors, its refusals and its idioms — which is why provenance of generated data is tracked rather than assumed.

Example: Why the filter matters more than the generator

A 2,000-example instruction set is expanded 5x to 10,000 by paraphrasing. If the paraphraser produces a label-inconsistent example 8% of the time, the 8,000 new examples contain 8,000 x 0.08 = 640 wrong ones, and the combined set of 10,000 carries a 640 / 10,000 = 6.4% noise floor it did not have before. Now add a label-consistency filter that catches 80% of the bad examples and wrongly discards 5% of the good ones: bad survivors are 640 x 0.2 = 128, good survivors are 7,360 x 0.95 = 6,992, so the set is 2,000 + 6,992 + 128 = 9,120 examples at 128 / 9,120 = 1.4% noise. The generator was unchanged; the filter moved the noise rate from 6.4% to 1.4% while costing 880 examples. This is why augmentation pipelines are judged by their rejection step, and why a reported 5x expansion with no rejection rate is not a result.

Commonly confused with

Augmentation and synthetic data generation are often used interchangeably but differ in where the content comes from. Augmentation transforms an existing labelled example, so the label is inherited and coverage grows around points you already have. Synthetic generation creates new examples from a model or a program, so the label must be established and coverage can extend into regions your data never reached — along with the generator's biases. The practical difference is the failure mode: augmentation risks correlated redundancy, synthetic generation risks model collapse from training on a model's own distribution.

When to use it

Reach for it when:

Labelled data is scarce and expensive relative to compute, which is the ordinary case for domain-specific fine-tuning.
A known robustness gap exists — the model fails on formatting variation, paraphrase, or entity substitution — and the transformation targets exactly that gap.
Class imbalance needs correcting on the minority class, where a few hundred real examples can support a usable expansion.
There is a cheap programmatic consistency check, so generated examples can be filtered rather than trusted.

Reach for something else when:

Without a filter. Unfiltered expansion reliably raises the noise floor, and the measured cost is a worse model from more data.
On the evaluation set, ever. It inflates the score and hides the regression it caused.
When plentiful real data is available. Augmented examples are correlated with their sources and are not substitutes for independent samples.
For transformations that can change the label — negation, numeric perturbation in arithmetic tasks, entity swaps in factual QA — where the operator manufactures wrong answers.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Data Augmentation — AI Glossary.

Frequently asked questions

What is Data Augmentation?