Counterfactual Explanation — AI Glossary

Reference details

Related terms

• Explainable AI (XAI)
• LIME (Local Interpretable Model-Agnostic Explanations)
• SHAP (SHapley Additive exPlanations)

How it works

1. The explanation is posed as a search rather than an inspection: find the nearest input to the one at hand that the model classifies differently.
2. Nearness needs a distance function, and the choice of metric is a modelling decision that determines which of many valid counterfactuals is returned.
3. Features are partitioned into actionable and immutable, so the search cannot propose changing age or ethnicity to obtain a different outcome.
4. Plausibility constraints keep the result on the data manifold, preventing a technically minimal answer that describes a person who could not exist.
5. The optimisation then minimises distance subject to the classifier crossing its decision boundary, typically by gradient descent for differentiable models or search for tree ensembles.
6. The output is returned as the change itself rather than as a feature importance, which is what makes it directly actionable by the person affected.

Example: Nearest in feature space is not nearest in effort

A lending model declines an application whose debt-to-income ratio is 0.42 against a threshold of 0.36, on an income of $48,000 and debts of $20,160 — and 20,160 / 48,000 = 0.42 confirms the starting point. Two counterfactuals cross the boundary. Raising income while debt is fixed requires 20,160 / 0.36 = $56,000, an increase of 56,000 - 48,000 = $8,000. Reducing debt while income is fixed requires 0.36 x 48,000 = $17,280, a reduction of 20,160 - 17,280 = $2,880. Both are valid, both are minimal along their own axis, and in dollar terms they differ by 8,000 / 2,880 ≈ 2.78 — so the debt path asks for just over a third of what the income path asks for. Which one a naive search returns depends entirely on the distance metric. An unweighted Euclidean distance over raw dollars picks the $2,880 debt reduction, which happens to be the right advice here. But normalise each feature by its standard deviation, as is conventional, and if income varies much more widely across the population than debt does, the $8,000 income change can look smaller in normalised units than the $2,880 debt change. The metric, not the model, then decides what the applicant is told to do — which is why the distance function in a counterfactual system is a policy choice that deserves the same review as the classifier.

Commonly confused with

A counterfactual explanation is not a feature-importance explanation, and they answer different questions. SHAP or LIME attribute the decision that was made, telling you which features drove it and by how much; a counterfactual describes a decision that was not made and what would reach it. Attribution is diagnostic and suits the model's developer, while a counterfactual is actionable and suits the person affected — knowing that debt-to-income contributed most to a denial is not the same as knowing that $2,880 of debt reduction reverses it. The second confusion is with causal inference. A counterfactual explanation is a statement about the model's decision surface, not about the world: it says what input the classifier would treat differently, and it carries no claim that making that change would cause the real-world outcome to improve.

When to use it

Reach for it when:

• For adverse-action notices and similar regulated explanations, where the recipient needs a route to a different outcome rather than a diagnosis.
• When the audience is the affected person rather than the model's developer, since the output is advice instead of attribution.
• Where an actionable-feature partition can be defined, which is what keeps the search from proposing immutable changes.
• Alongside attribution methods, since the two answer complementary questions about the same decision.

Reach for something else when:

• As a causal claim about the world, which a statement about a decision boundary does not support.
• Without plausibility constraints, where the minimal answer can describe a combination of features no real applicant has.
• With an unreviewed distance metric, since the metric chooses among valid counterfactuals that differ severalfold in real effort.
• For debugging a model, where feature attribution over the training distribution is the more direct instrument.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Counterfactual Explanation — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path