Certified, Not Explained: Exact Token Erasure in Gated Vision State Space Models

Sadia Parveen, Tehseen ZiaarXiv 2026 · arXiv

In the quest for interpretable AI, we often rely on saliency maps—those colorful heatmaps that supposedly show "where the model is looking." But as many researchers have pointed out, these maps can be deceptive, looking plausible to humans while being unfaithful to the model's actual logic.

A new paper, "Certified, Not Explained: Exact Token Erasure in Gated Vision State Space Models," by Sadia Parveen and Dr. Tehseen Zia, offers a more rigorous alternative. Instead of just asking where a model looks, they provide a mathematical guarantee of what a model cannot see. By building on the emerging "Mamba" architecture (Selective State Space Models), they've developed a framework for exact token erasure: a way to prove that specific parts of an image have zero influence on the final prediction.

TL;DR

The authors introduce CG-Mamba, a vision model that provides a "certified support set"—a mathematically guaranteed list of image patches that contribute to a prediction. They prove that by gating the input to a Mamba block's convolution, they can achieve "exact erasure," where changing "closed" tokens results in exactly zero change to the model's output.

What Problem Does This Paper Address?

The paper tackles two major headaches in medical AI:

  1. Unfaithful Explanations: Traditional saliency methods (like Grad-CAM) often fail "sanity checks," showing the same heatmap even if the model's weights are randomized.
  2. Concept Leakage: In Concept Bottleneck Models (CBMs)—which predict human-readable traits (like "spiculation" in a lung nodule) before the final diagnosis—hidden information about the label can "leak" through the concept layer, making the model less transparent than it seems.

The authors specifically look at Mamba-based models, which are often praised for their "selection" mechanism. They wanted to know: can we turn this selection into a hard guarantee for interpretability?

Methods

The researchers built CG-Mamba, a CBM for lung nodule malignancy. Its core is a single gated, bidirectional selective state-space (S6) block.

The CG-Mamba architecture: Image patches are gated before entering a bidirectional S6 block, which then feeds into partitioned concept heads.

Key technical innovations include:

  • The Backbone Gate: A learned mask that "closes" certain image tokens.
  • Exact Erasure Conditions: They mathematically identified that to truly erase a token's influence, you must mask the input to the block's causal convolution and ensure closed tokens have zero weight at the readout stage.
  • Readout Partitioning: They split the model's hidden channels into dedicated groups for each clinical concept to minimize information leakage between them.
  • Path-by-Path Knockout: A rigorous audit procedure that corrupts "closed" tokens with noise to verify that the output remains unchanged.

Datasets

The study uses the LIDC-IDRI dataset, a gold standard for lung cancer screening. It contains CT scans of pulmonary nodules along with radiologist ratings for six concepts: spiculation, margin, sphericity, lobulation, subtlety, and texture.

Key Insights and Ideas

  • $\Delta$ is a Red Herring: In Mamba literature, the selective step size ($\Delta$) is usually cited as the selection mechanism. This paper surprisingly finds that gating $\Delta$ is neither necessary nor sufficient for exact erasure. The real gatekeeper is the input to the convolution.
  • Certified vs. Explained: The authors argue we should view these gates not as "explanations" (which imply human-like reasoning) but as "certified support sets." It’s a structural guarantee: "I promise these specific pixels did not affect the result."
  • Leakage is Deceptive: By decomposing the standard Oracle Impurity Score (OIS), they found that much of what looks like "leakage" is actually just the model being good at predicting a concept from its own features, rather than "cheating" by looking at other concepts.

Key Results

The "knockout" experiments provided striking binary results. When the authors' conditions for erasure were met, the change in the model's output was exactly 0.0000. When they violated even one condition (like removing the convolution mask), the output jumped significantly.

Table 1: Leakage-path knockout on the final model: 15 runs (5 folds × 3 seeds), held-out folds; AUROC is mean ± SD over the 15 runs. Corruption (Gaussian noise with SD 50, or air) touches closed tokens only, with the gate pattern forced. “Exact” means a maximum change of 0 at both the concept and the task logits in that run. The AUROC column switches a path off at evaluation time in a model trained with all gates; it measures train/test mismatch and is not the accuracy that a model trained without that path would reach. Pre-declared K1 (full, no-∆ and conv-only exact in every run): pass. K2 (no conv mask, and conv-only with gj-only pooling, broken): pass.

Quantitatively, CG-Mamba holds its own:

  • Accuracy: It achieved a pooled AUROC of 0.8425, virtually identical to an ungated Mamba baseline (0.8428).
  • Leakage: Partitioning the readout reduced "excess" cross-concept predictability by about 0.024 AUROC points, successfully tightening the bottleneck.
Performance comparison showing CG-Mamba (blue) is competitive with other Mamba and ViT-based concept bottleneck models.

Limitations and Future Work

While the guarantee is mathematically "exact," the authors are careful to note its limits:

  • Narrow Scope: The guarantee applies to a fixed gate pattern. If you change the image enough to change the gate itself, the guarantee must be re-evaluated.
  • Localization vs. Logic: Label-randomization tests showed that the gates mostly reflect where the nodule is (localization) rather than the specific "logic" of malignancy.
  • Scaling to 3D: Initial attempts to move from 2D slices to full 3D volumes underperformed, suggesting that while the theory holds, 3D Mamba models require more tuning and compute to be practical for this task.

This work moves the needle for high-stakes AI by providing a "checkable" form of interpretability—one where a clinician can verify, for any given image, exactly which parts of the scan were ignored.

Summary updated

Abstract

Vision models built on Mamba are often explained through their selective step size Delta, on the assumption that a token the scan does not admit cannot influence the prediction. We test that in a concept bottleneck model for lung-nodule malignancy built on a single gated, bidirectional selective state-space block. The answer becomes an interpretability framework: a guarantee, a procedure that checks it, and a pre-registered audit of what it explains. For a fixed gate pattern, masking the input of the causal convolution of the block and giving closed tokens zero weight at readout makes the concept and task logits exactly invariant to the closed tokens. Gating Delta is neither necessary nor sufficient. On LIDC-IDRI, a path-by-path knockout over 15 cross-validated models agrees with this in every run: the largest logit change is exactly 0 whenever the conditions hold and between 4.1 and 6.7 when they do not. The gated model reaches a pooled out-of-fold AUROC of 0.8425 +/- 0.0085, level with an ungated Mamba bottleneck (0.8428). The guarantee belongs to the masked model, not to the gate as an explanation. Post-hoc rankers, which inherit the guarantee on this model, order tokens better than the gate. Gate maps from a model trained on permuted labels are as close to the real maps as two real-label seeds are to each other. Under retraining, the tokens of the gate carry more task information than random tokens and, on shifted crops, more than central ones, but no more than the tokens chosen by hidden attention. We therefore read the gate as a certified support set produced by mask supervision. Splitting the Oracle Impurity Score into own-concept and cross-concept parts reveals differences the aggregate score misses. Partitioning the readout reduced leakage slightly under an earlier recipe, and the final model sits at, rather than below, the level implied by correlations among the true concepts.
BibTeX
@misc{parveen2026certified,
  author = {Sadia Parveen and Tehseen Zia},
  title = {{Certified, Not Explained: Exact Token Erasure in Gated Vision State Space Models}},
  year = {2026},
  eprint = {2611.20897},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20897},
}
Figure