ModelRefs / LIME (Local Interpretable Model-Agnostic Explanations) — AI Glossary

LIME (Local Interpretable Model-Agnostic Explanations) — AI Glossary

An explainability method approximating a black-box model locally with an interpretable surrogate to explain individual predictions.

Overview

LIME (Ribeiro et al. 2016) perturbs the input around an instance, queries the black-box model, and fits a simple interpretable model (linear, decision tree) to the local neighborhood. For text: identifies which tokens most influenced the prediction. Model-agnostic: works for any classifier or LLM. Computationally expensive for long texts.

Reference details

Topic	safety
Last reviewed	2026-06-24

How it works

The instance is perturbed by removing or masking features — words, for text — producing a neighbourhood of variants around the original input.
Each variant is passed through the model and its output recorded, so the method needs only query access and never gradients or weights.
Variants are weighted by how close they remain to the original under a similarity kernel, so a heavily corrupted variant contributes less to the fit than a lightly changed one.
That kernel has a width parameter, and it is a free choice: a narrow kernel explains behaviour in a tiny neighbourhood while a wide one describes something closer to the model's global behaviour.
A sparse linear model is then fitted to the weighted samples, with the sparsity constraint forcing credit onto a few features so the explanation is readable.
Both the perturbation sample and the kernel width therefore enter the result, which is why an explanation is reproducible only when the sample count, the seed and the kernel are all recorded.

Example: Count the model calls before you commit

LIME explains one prediction by perturbing the input — dropping words or features — and asking the model what changes. Five hundred perturbations means five hundred inference calls to explain a single prediction. Explaining a thousand predictions for an audit is half a million calls. And because the perturbations are sampled randomly, running the same explanation twice can return different top features unless the sample is large enough or the seed is fixed. Report the perturbation count and the seed, or the explanation is not reproducible.

Commonly confused with

LIME and SHAP both attribute a prediction to input features, but not on the same basis. LIME fits an interpretable surrogate to the local neighbourhood — fast, intuitive, and sensitive to how that neighbourhood was sampled. SHAP computes Shapley values with consistency guarantees, at higher cost. Disagreement between them is common, and is a signal to distrust both.

When to use it

Reach for it when:

Explaining individual predictions of a black-box classifier to a non-technical reviewer
Model-agnostic settings where you have query access but not gradients or weights
Rapid hypothesis generation about what a model is keying on

Reach for something else when:

As evidence of causation — it explains the surrogate's behaviour near one point, not the model's reasoning
Long documents or high-dimensional inputs, where cost and instability both climb
Compliance artefacts requiring reproducibility, unless sampling is fixed and documented

Primary source

Ribeiro et al., "Why Should I Trust You?": Explaining the Predictions of Any Classifier (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to LIME (Local Interpretable Model-Agnostic Explanations) — AI Glossary.

Frequently asked questions

What is LIME (Local Interpretable Model-Agnostic Explanations)?