Explaining Black Box Models with Local Surrogates

344 concepts248 papers3 proposals

Deep neural networks, ensemble methods, and large language models often achieve high predictive accuracy at the cost of interpretability. Globally, these models construct highly complex, non-linear decision boundaries in high-dimensional spaces. Trying to explain the entire global behavior of such a model all at once is often an intractable task.

Local surrogate modeling bypasses this complexity by shifting the focus from how the model works everywhere to how the model works right here. The core intuition relies on a simple geometric principle: any smooth, highly non-linear boundary looks approximately flat if you zoom in close enough to a single data point. By capturing this local linearity, we can explain individual predictions faithfully without needing to comprehend the entire global model.

The Core Intuition: Local Flatness

Imagine trying to map the topography of the Earth. Globally, the planet is a complex oblate spheroid with jagged mountain ranges and deep trenches. However, if you stand on a single street corner, the ground beneath you behaves as a flat, two-dimensional plane. You can navigate your immediate surroundings perfectly using simple linear directions (e.g., "go three steps forward and two steps left").

As Wojciech Samek et al. discuss in Towards Explainable Artificial Intelligence, complex machine learning models behave similarly. While the global model is a black box, its behavior within a restricted region around a specific input can be accurately mirrored by an inherently interpretable "surrogate" model, such as a sparse linear regression or a shallow decision tree. As Andreas Holzinger notes in Explainable AI Methods - A Brief Overview, this local surrogate yields human-readable explanations—like linear coefficients or logical rules—that are valid within that specific neighborhood.

The Mechanism: Perturb, Predict, Weight, and Fit

To build a local surrogate model without knowing the internal architecture or weights of the black box, researchers treat the primary model as an oracle. In Sampling, Intervention, Prediction, Aggregation: A Generalized Framework for Model-Agnostic Interpretations, Christian A. Scholbeck et al. formalize this process into a structured pipeline:

  1. Perturb (Sampling & Intervention): Take the specific instance you want to explain and generate a synthetic dataset around it by introducing small variations. For a text model, this might mean deleting words; for a tabular model, it means adding small amounts of noise to numerical features.
  2. Predict: Pass these perturbed samples through the black-box model to observe how the outputs change.
  3. Weight: Measure the distance between each perturbed sample and the original instance. Assign higher weights to samples that are close to the original point and lower weights to those further away. This ensures the surrogate model prioritizes local accuracy over global fit.
  4. Fit (Aggregation): Train a simple, transparent model (like a ridge regression) on this weighted, perturbed dataset.

As Anders Gjølbye et al. explain in Minimizing False-Positive Attributions in Explanations of Non-Linear Models, the resulting discriminative weights of the linear surrogate represent how much each feature contributes to the model's prediction in that specific slice of the input space.

[Original Input] ──> Generate Perturbations ──> [Weighted Neighborhood]
                                                          │
   ┌──────────────────────────────────────────────────────┘
   ▼
[Black-Box Model] ──> Get Predictions ──> Train Linear Surrogate ──> [Feature Importance]

Adapting to Different Data Modalities

Because this framework is model-agnostic, it can be adapted to virtually any data modality by changing how the "neighborhood" is constructed and how features are represented.

Text and Language

In natural language processing, a local surrogate can pinpoint exactly which words or phrases triggered a classification. For instance, Abdessamad Benlahbib et al. utilize local explanations in LISAC FSDM USMBA at SemEval-2021 Task 5: Tackling Toxic Spans Detection Challenge with Supervised SpanBERT-based Model and Unsupervised LIME-based Model to highlight influential words that contribute to toxicity predictions.

However, randomly deleting words can result in ungrammatical nonsense that confuses the underlying model. To fix this, Mahtab Sarvmaili et al. introduce a framework in A Modularized Framework for Explaining Black Box Classifiers for Text Data that uses linguistic ontologies to generate realistic, semantically related variations of a document, ensuring the local neighborhood remains natural.

Images and Latent Spaces

For computer vision, instead of perturbing raw pixels, explainers often operate on a higher level of abstraction. Carlo Metta demonstrates this in Exemplars and Counterexemplars Explanations for Skin Lesion Classifiers by mapping images into a compressed latent space. The author samples the neighborhood within this latent space and trains a local decision tree to extract rules, generating synthetic "exemplars" (similar images with the same diagnosis) and "counterexemplars" (similar images that flip the diagnosis) to map out the local decision boundary visually. Yuan Yin similarly leverages this latent neighborhood approach in Exemplars and Counterexemplars Explanations for Image Classifiers, Targeting Skin Lesion Labeling to extract local decision rules for medical imaging.

Complex Architectures: Graphs and Time-Series

Local surrogates can also decouple explanations from highly complex structural architectures. In Distill n' Explain: explaining graph neural networks using simple surrogates, Tamara A. Pereira et al. show that distilling a Graph Neural Network's local behavior into a simple linear surrogate bypasses the need to backpropagate through complex graph topologies, accelerating explanation generation by orders of magnitude. For sequential data, Owen Queen et al. develop methods in Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency to train surrogates that preserve temporal consistency, revealing which specific time-varying patterns triggered a decision.

The Fragility of the Neighborhood

While local surrogate modeling is highly intuitive, its success depends entirely on how the local neighborhood is defined and sampled. If the perturbations are too broad, the surrogate model will attempt to fit a non-linear space with a linear function, leading to unfaithful explanations.

Furthermore, the choice of how to "mask" or remove features can fundamentally alter the results. Saumitra Mishra et al. demonstrate this vulnerability in Reliable Local Explanations for Machine Listening, showing that the way you define "silence" when masking audio spectrograms drastically changes which features the surrogate deems important.

Different local explanation paradigms can also disagree on the same data point. Damien Garreau et al. prove in Comparing Feature Importance and Rule Extraction for Interpretability on Text Data that feature importance methods and rule extraction methods often diverge because they navigate and simplify the local decision space differently.

Finally, as Jihan K. Zaki et al. emphasize in Explainable Deep Learning Framework for SERS Bioquantification, users must remember that a local surrogate explains the approximation of the model's local behavior, rather than providing a direct window into the exact internal mechanics of the underlying black box. Despite these challenges, local surrogates remain a foundational tool for translating high-dimensional algorithmic decisions into localized, human-interpretable narratives.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_local_surrogate_modeling_local_interpret_f2e71ff8,
  title = {local surrogate modeling (local interpretable model-agnostic explanations)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-surrogate-modeling-local-interpretable-model-agnostic-explanations/rdr_f2e71ff893f74732863860e766e40d2e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_local_surrogate_modeling_local_interpret_f2e71ff8_v1,
  title = {local surrogate modeling (local interpretable model-agnostic explanations)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-surrogate-modeling-local-interpretable-model-agnostic-explanations/rdr_f2e71ff893f74732863860e766e40d2e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake