ModelRefs / AI Anomaly Detection — AI Glossary

AI Anomaly Detection — AI Glossary

Using ML to identify unusual patterns deviating from expected behavior in time series, logs, images, or text.

Overview

LLM-based anomaly detection: classify log lines as anomalous via prompting, detect semantic drift in text streams, and identify unusual patterns in structured data described in natural language. Traditional ML (Isolation Forest, LSTM autoencoders) still dominates for high-volume metrics; LLMs add interpretability by explaining detected anomalies.

Reference details

Topic	applications
Last reviewed	2026-06-24

How it works

A model of normal is fitted first, either statistically over a feature window or as a reconstruction objective that learns to reproduce ordinary inputs accurately.
Each new observation is scored by its distance from that model — reconstruction error, density under the fitted distribution, or isolation depth in a random partition.
A threshold converts the continuous score into an alert, and the choice of threshold is the entire operating characteristic of the system.
Because anomalies are rare, the threshold interacts with the base rate rather than with the score distribution alone, which is where most deployments go wrong.
Alerts are grouped and deduplicated before reaching a human, since a single underlying event usually trips many correlated signals.
Language models enter at the explanation stage rather than the detection stage, turning a flagged record into a readable account of why it looked unusual.

Example: The base rate decides whether alerts are usable

Take 10,000,000 log lines a day with a genuine anomaly rate of 0.01%, so 1,000 real anomalies and 9,999,000 ordinary lines. Deploy a detector with 95% recall and a 1% false-positive rate — figures that sound entirely respectable. It catches 1,000 x 0.95 = 950 of the real anomalies and raises 9,999,000 x 0.01 = 99,990 false ones, so 100,940 alerts in total of which 950 are real. Precision is 950 / 100,940 ≈ 0.94%, meaning about 99 in every 100 alerts are noise and no on-call rotation will survive the week. Fixing this is a demand on the false-positive rate specifically, and the required movement is severe. To reach 50% precision the detector must raise no more false alerts than true ones, so about 950 of them, which needs a false-positive rate of 950 / 9,999,000 ≈ 0.0095%. Against the 1% it started at, that is a tightening by a factor of about 105. Recall barely matters by comparison: dropping from 95% to 80% recall loses 150 real anomalies and moves precision from roughly 0.94% to 0.80 x 1,000 / (800 + 99,990) ≈ 0.79%, so it makes matters slightly worse rather than better. The lever is the false-positive rate, and the reason is that it multiplies the nine-million-line denominator while recall multiplies the thousand-line numerator.

Commonly confused with

Anomaly detection is not classification, and the difference is what the training data contains. A classifier learns from labelled examples of both classes and can be evaluated on held-out labels; an anomaly detector learns only what normal looks like, because the anomalies it most needs to catch are the ones nobody has seen yet. This is why anomaly detection is the right tool for novel failure modes and the wrong one for known ones — once a failure type has enough labelled examples, a classifier beats a density estimate on it. It is also distinct from alerting on a threshold, which is a fixed rule over a known quantity; an anomaly detector fits what normal is from data, which means it drifts as the system changes and will quietly learn to consider a slow degradation normal.

When to use it

Reach for it when:

For novel failure modes with no labelled examples, which is the case supervised classification cannot address at all.
Where the cost of a missed event is high and a human can triage a manageable alert volume.
As a first-pass filter ahead of a classifier, once enough of the flagged events have been labelled to train one.
With a language model layered on explanation rather than detection, which is where it adds interpretability without touching the base-rate arithmetic.

Reach for something else when:

At very low base rates without a false-positive rate to match, since 1% false positives on ten million records produced about 0.94% precision above.
For failure modes that already have labels, where a trained classifier outperforms a model of normality.
On systems that change constantly, where the fitted normal drifts along with the degradation it should be catching.
Where no triage capacity exists, because an alert nobody investigates is a cost with no detection benefit.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to AI Anomaly Detection — AI Glossary.

Frequently asked questions

What is AI Anomaly Detection?