Cutpurse: Decomposing Explanation-Guided Model Extraction on Tabular Data

Seyma Selcan Magara, Hanru Zhang, Fatme El Zahraa Hamad, Nico Pfeifer, Mete AkgünarXiv 2026 · arXiv

TL;DR

This paper challenges the long-held belief that explanation APIs (like LIME or SHAP) are the primary culprits in model extraction attacks. By decomposing these explanations, the authors show that the "secret" information they leak is often easily reproducible from public data, and they introduce CUTPURSE—an attack that achieves state-of-the-art extraction fidelity without ever calling an explanation API.

What Problem Does This Paper Address?

Machine Learning as a Service (MLaaS) providers often return "post-hoc explanations" alongside predictions to improve transparency. However, security researchers worry these explanations act as a side channel, helping adversaries "steal" the model's logic through model extraction.

The core question this paper tackles is: Does an explanation API actually reveal model-specific secrets, or does it just provide a convenient "cheat sheet" for query placement? Previous research showed that explanation-guided attacks are effective, but they didn't isolate which part of the explanation—the feature rankings (attribution) or the specific values used for probing (coordinates)—was doing the heavy lifting.

Methods

The authors first perform a "channel decomposition" of existing attacks. They separate an explanation into:

  1. The Attribution Channel: Ranks which features are most influential.
  2. The Coordinate Channel: Supplies candidate feature values (like LIME's discretization bin edges) for perturbing those features.

They discovered that while bin edges are highly useful, they are often just statistics of the background data, not the model itself. This led to the development of CUTPURSE, a three-phase, explanation-free attack:

  • Phase 1: Diverse Query Generation: Uses crossover and mutation heuristics to explore regions of the input space far from the initial seed data.
  • Phase 2: Boundary Search: Actively searches for the decision boundary between samples of different classes using bisection.
  • Phase 3: Self-Grid Traversal: Instead of asking an API for bin edges, CUTPURSE fits its own discretization grid using a small amount of auxiliary data. It then perturbs features based on this "self-grid."
Fig. 1. The attacker fits a grid on auxiliary data, then uses diverse generation, boundary search, and self-grid traversal to construct a surrogate dataset. It queries the prediction API but does not query the explanation API, the greyed-out explanation endpoint is shown for context only

Datasets

The study evaluates attacks across six public tabular datasets covering continuous, categorical, and mixed features:

  • Crop: 17 classes, continuous features.
  • Adult: Income prediction, mixed features.
  • Breast Cancer: Binary classification, continuous features.
  • Nursery & Mushroom: Purely categorical features.
  • Pendigits: Handwritten digits, continuous features.
Summary of datasets used in the evaluation.

Key Insights and Ideas

  • Attribution is a Weak Signal: Surprisingly, using feature rankings to decide which features to perturb provides no systematic advantage over just picking features at random.
  • Bin Edges are Query-Placement Aids: LIME’s discretization bin edges are the real "secret sauce" for high-fidelity extraction, but they aren't actually secrets. They are quantiles of the data used to fit the explainer.
  • The "Leak" is Reproducible: An attacker can fit their own grid on a tiny amount of auxiliary data (sometimes as little as one sample per class) and achieve the same performance gains as if they had queried the service's explanation API.
  • Smarter Traversal > Explanations: By focusing on diverse exploration and boundary searching, CUTPURSE proves that an adaptive attacker can recover better query-placement capabilities independently of any explanation API.

Key Results

  • Grid Parity: In controlled tests, an attacker-fitted grid reproduced the fidelity gains of the service's grid almost perfectly. The "target advantage" (the benefit of using the service's grid over the attacker's) was negligible.
  • Fidelity Boost: Against a "defended" service that withholds bin edges, CUTPURSE improved extraction fidelity by 16.3 percentage points on continuous-feature targets.
  • Outperforming the State-of-the-Art: Even when compared to the original Autolycus attack (which uses explanation APIs), CUTPURSE achieved 7.0 percentage points higher fidelity by using its superior traversal strategy.
Comparison of LIME-ranked vs random feature selection, showing that ranking provides no consistent benefit.

Limitations and Future Work

The authors acknowledge several boundaries to their findings:

  • Tabular Focus: The results are specific to tabular data and "classical" ML models (Decision Trees, Logistic Regression, etc.). They do not necessarily apply to high-dimensional image or text data or deep neural networks.
  • Explainer Configurations: The study focuses on default LIME and SHAP configurations. Different discretizers or explanation methods (like counterfactuals) might still leak privileged information.
  • Categorical Data: CUTPURSE's grid-based approach is designed for continuous features; it offers no advantage on purely categorical datasets where "bin edges" have no natural meaning.
  • Future Work: A key open question is identifying which specific explanation outputs cannot be reproduced by an adaptive adversary, as these would represent the true security risk of XAI.

Summary updated

Abstract

Explanation APIs are often treated as increasing model-extraction risk, but an end-to-end attack comparison cannot show whether an explanation reveals model-specific information or merely supplies a convenient next-query coordinate. We study this distinction for explanation-guided extraction of tabular classifiers. We separate an explanation into an attribution channel, which ranks influential features, and a coordinate channel, which supplies candidate feature values for perturbing those features. We then test whether the useful bin edges are privileged to the service or reproducible from the adversary's own data. Across six datasets and five target families, using attribution rankings to choose which features to perturb provides no systematic extraction advantage. Lime's discretization bin edges can improve fidelity by up to 41 percentage points on continuous features, but those edges are quantiles of the explainer's background data rather than properties of the target model. In controlled comparisons, an attacker-side grid reproduces the useful bin-edge gains without querying an explanation API. We use these findings to derive Cutpurse, which combines a self-computed grid with diverse query generation and boundary search. Against a service that withholds bin edges, Cutpurse improves fidelity by 16.3 percentage points on continuous-feature targets. It also exceeds the undefended published attack by 7.0 percentage points. Thus, withholding default Lime bin edges alone does not prevent effective extraction under the in-domain auxiliary-input, probability-output, and same-family surrogate assumptions studied here. Our results concern default Lime and Shap configurations for tabular classifiers. They do not establish the same conclusion for other explainers, target classes, or data-free attackers.
BibTeX
@misc{magara2026cutpurse,
  author = {Seyma Selcan Magara and Hanru Zhang and Fatme El Zahraa Hamad and Nico Pfeifer and Mete Akgün},
  title = {{Cutpurse: Decomposing Explanation-Guided Model Extraction on Tabular Data}},
  year = {2026},
  eprint = {2611.09375},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09375},
}
Figure