SHAP (SHapley Additive exPlanations) — AI Glossary

Reference details

Related terms

• LIME (Local Interpretable Model-Agnostic Explanations)
• Interpretability
• Explainable AI (XAI)

How it works

1. A feature's contribution is defined as its average marginal effect on the prediction across every possible ordering of the features, which is what gives the method its consistency guarantees.
2. Computing that requires evaluating the model with features absent, and a model cannot take an absent feature — so absence has to be simulated.
3. Absence is simulated by substituting values drawn from a background distribution and averaging over them, which means the choice of background set is part of the computation rather than a detail of it.
4. A background of all-zeros, a sample of the training data, or a cluster of similar instances each produce different attributions for the identical prediction and model.
5. The number of orderings is exponential in the feature count, so only structured models such as tree ensembles admit an exact algorithm and everything else is sampled.
6. Sampling reintroduces the run-to-run variance the method's guarantees were supposed to remove, which is why a sampled attribution needs its background set and sample count reported alongside it.

Example: The attributions have to add up, and that is the point

The defining property is additivity: the feature contributions sum exactly to the prediction minus the base value. If the average prediction is 0.30 and this instance scores 0.82, the contributions must total 0.82 − 0.30 = 0.52 — say +0.30, +0.15, +0.09 and −0.02. That is a real constraint, and it is what makes the attributions comparable across features and across instances. Local surrogate methods offer no such guarantee, which is why their explanations can shift between runs. The cost is computation: exact values consider all feature subsets, so anything beyond tree models is an approximation with its own sampling error.

Commonly confused with

SHAP and LIME both attribute a prediction to features, on different foundations. SHAP computes Shapley values with consistency and additivity guarantees at higher cost; LIME fits a local surrogate — faster, and sensitive to how its neighbourhood was sampled. Where they disagree, the disagreement is information: it usually means the local behaviour is not well approximated by either simple account.

When to use it

Reach for it when:

• Tabular models, especially trees, where the exact algorithm is fast
• Regulated settings needing a defensible, reproducible attribution method
• Global feature importance built by aggregating local attributions

Reach for something else when:

• High-dimensional text or images without accepting a sampling approximation
• As a causal claim — attribution describes the model's behaviour, not the world
• Where correlated features make individual credit assignment inherently ambiguous

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to SHAP (SHapley Additive exPlanations) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path