AI Test Generation — AI Glossary

Reference details

Related terms

• Code Generation
• AI Code Review
• Coding Agent

How it works

1. The target code is supplied as context along with its imports, type signatures and nearby test conventions, since a generated test must compile against the real module and match the project's existing framework and style.
2. The model proposes cases by reading structure: branches, boundary conditions, error paths and null handling. It is inferring intent from implementation, which is the mechanism and also the central limitation.
3. Each case needs setup — fixtures, mocks, fakes for I/O — and generating these correctly is usually harder than generating the assertions, because it requires knowing which collaborators are safe to fake.
4. Assertions are written from what the code currently does. A test asserting the present behaviour of a buggy function passes and locks the bug in.
5. Generated tests are executed, and failures are triaged into two piles: the test is wrong, or the code is. Only a human can sort that pile, because the distinction depends on intent the code does not state.
6. Coverage is measured as the selection signal, which is why generation is strongest on branch coverage of pure functions and weakest everywhere behaviour depends on something outside the file.

Example: Why coverage delta overstates the gain

A module has 400 lines, 40% covered — 160 lines. Generated tests raise coverage to 85%, or 340 lines, a gain of 180 lines and a very good-looking number. Suppose the generator infers intent correctly 85% of the time on this codebase: of the new tests covering those 180 lines, roughly 180 x 0.15 = 27 lines are covered by tests asserting behaviour that was never intended. Those tests pass today, so they are invisible, and they will fail on the next correct refactor — converting 27 lines of coverage into 27 future false alarms. The honest reading is that coverage measures lines executed, not behaviour verified: the 180 lines are real execution, the 27 are anti-regression traps, and sorting them costs a review of each generated test. That review is the actual work, and the generator's value is that writing 180 lines of scaffolding by hand would have cost more than reading it.

Commonly confused with

Generated tests verify current behaviour, not intended behaviour, and this is not a quality issue to be fixed by a better model — it is structural. A specification-derived test can fail on correct code because the code is wrong; an implementation-derived test cannot, because it was written from that code. So generated tests are regression tests by construction: excellent for detecting change, useless for detecting that the original was wrong. A suite of them at 90% coverage says the behaviour is pinned, not that it is correct, and reporting the second from the first is the error that matters.

When to use it

Reach for it when:

• Backfilling tests on untested legacy code, where pinning current behaviour before refactoring is exactly what you want.
• Boilerplate and scaffolding — fixtures, parametrised cases, mock setup — where the volume is the cost and the content is mechanical.
• Pure functions with clear inputs and outputs, where branch and boundary enumeration is where generation is strongest.
• Finding gaps: generated cases often surface an unhandled error path a human skipped, which is useful even if the test itself is rewritten.

Reach for something else when:

• As verification of correctness for new code, where the test is derived from the implementation it is meant to check.
• For specification-driven work — a written spec, a contract, a regulatory requirement — where the test must come from the spec, not the code.
• Merged without review. Unreviewed generated tests accumulate assertions nobody chose and fail on correct future changes.
• As a coverage target to satisfy. Coverage generated to hit a number produces tests that execute lines and assert nothing meaningful.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to AI Test Generation — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path