Keeping Parts Honest: Isolated Part Graphs for Faithful Vision Transformer Explanations

Bernard Tomczyk, Gail Collyer-Hoar, Bryan WilliamsarXiv 2026 · arXiv

TL;DR

IsoPartFormer is a novel Vision Transformer (ViT) architecture that ensures part-based explanations are "honest" by encoding image regions in total isolation, preventing the "evidence leakage" that compromises existing interpretable models. It achieves state-of-the-art accuracy across 11 benchmarks while providing mathematically faithful justifications where every bit of evidence is strictly tied to the image patches it claims to represent.

What Problem Does This Paper Address?

In the quest for "interpretable" AI, researchers have developed part-based classifiers that explain a decision by pointing to specific image regions—for example, "this is a Woodpecker because of its beak." However, Vision Transformers (ViTs) rely on self-attention, a mechanism that mixes information across the entire image.

The authors identify a critical flaw: "evidence leakage." Because tokens in a ViT attend to all other patches, the feature representing the "beak" often absorbs context from the "tail" or the "wings." Consequently, the model's explanation is unfaithful; it claims to use the beak, but it’s actually using a feature that "knows" about the whole bird. The authors show that when you force existing models to look only at their explained parts (an "isolation audit"), their accuracy collapses, proving they rely heavily on this hidden context.

Figure 1. Our approach discovers unannotated coarse and fine sub- parts, encodes each in isolation and connects pairs by edges. The resulting prediction is an exact sum of node z(k) and edge z(uv)

Methods

The authors propose IsoPartFormer, a classifier designed to be "region-faithful" by construction. Its key components include:

  1. Isolation Audit: A new testing protocol that evaluates a model by encoding each explained region in isolation (removing all other patches). This measures how much a model's performance depends on leaked context.
  2. Isolated Part Encoding: Instead of one global attention map, IsoPartFormer treats the image as a graph. Each discovered part is encoded as a separate token stream. These streams never "talk" to each other in the backbone, ensuring a part's feature is derived strictly from its own pixels.
  3. Explicit Interaction Edges: To recover useful context without reintroducing leakage, the model uses explicit edges between pairs of parts. These edges are computed only from the two parts they connect and are credited as a distinct term in the final decision.
  4. Hierarchical Part Discovery: The model automatically discovers parts (e.g., "head," "torso") and sub-parts without human annotations by leveraging self-supervised features from DINOv2.
  5. Prototype Graph Matching: The final prediction is a direct sum of node (part) and edge (pair) similarities to learned class prototypes. This makes the logit mathematically decomposable into the exact contributions of each part and pair.
Figure 2. Overview of IsoPartFormer. (a) Parts discovered without annotations, as coarse parts divided into sub-parts, each pass alone through the frozen ViT blocks (cached once) and the fine-tuned last blocks, giving node descriptors fk, while each edge passes the cached tokens of exactly two parts through the fine-tuned blocks. Matching node and edge descriptors to each class’s prototype graph gives terms that sum exactly to the logit zc, and every term is also trained to predict the class on its own. (b) The explanation reads each part’s and pair’s share of the margin directly from these terms. (c) Our audit compares a region’s evidence on x, on xn (all other regions removed) and on x−n (the region itself removed).

Datasets

The model was evaluated on a massive suite of 11 benchmarks, covering both fine-grained and general object recognition:

  • Fine-grained: CUB-200-2011 (Birds), FGVC-Aircraft, Stanford Cars, Oxford Flowers-102, Oxford-IIIT Pets, NABirds, and Stanford Dogs.
  • General-object: STL-10, PASCAL VOC (object crops), PartImageNet, and ImageNet-S50.

Key Insights and Ideas

  • The "Collapse" of Interpretability: Existing state-of-the-art interpretable ViTs lose between 33 and 82 points of accuracy when forced to operate under isolation. This suggests that their "explanations" are often post-hoc justifications rather than a true reflection of the model's internal logic.
  • Faithfulness by Construction: By restricting information flow, IsoPartFormer guarantees that if a part is masked, its contribution to the decision drops to near zero. In contrast, existing models still credit masked parts with up to 95% of their original evidence due to leakage.
  • Context is Better as an Edge: The authors show that you don't need global self-attention for high accuracy. Restricting interactions to part-pairs (edges) is sufficient to recover the context needed for complex recognition tasks while keeping the explanation clean.
Figure 3. Share of a part’s evidence that a model still credits to it after the part’s pixels are replaced by the mean colour (top, part; bottom, masked). Evidence that truly comes from the part should vanish when it is masked, so lower is better. Existing models keep 75–95%, which must come from the rest of the image, whereas ours keeps 3%, which isolation guarantees comes from the mean- colour region itself rather than from other parts.

Key Results

  • Superior Accuracy: IsoPartFormer exceeds the accuracy of all existing "leaky" explainable methods on 10 out of 11 benchmarks when tested under isolation. On CUB-200, it achieves 91.9% accuracy while maintaining perfect region faithfulness.
  • High Faithfulness Correlation: The model shows a 0.94 rank correlation between a part's claimed contribution and the actual effect on the output when that part is removed. Competing methods scored at most 0.34.
  • Human-Centric Validation: In a pilot study, humans were significantly better at predicting IsoPartFormer's decisions compared to other methods, suggesting the explanations are more intuitive and trustworthy.
Figure 4. Explanation of our model for a Cliff Swallow (CUB [73]). Left, discovered parts, with pair edges drawn in proportion to their contribution. Middle, training image of the predicted class with a similar part layout. Right, contribution of each part and of the strongest pairs, with each part shown next to the closest same parts in training images of the class.

Limitations and Future Work

  • The "Price" of Faithfulness: While IsoPartFormer is highly accurate, it loses a small amount of performance (0.7 to 3.2 points) compared to a "leaky" version of itself with full attention. This is the inherent trade-off for strictly honest explanations.
  • Part Discovery Reliance: The model's performance depends on the quality of the initial unsupervised part discovery. If the parts don't align well with semantic features, the explanations may be less useful to humans.
  • Future Directions: The authors suggest that the isolation audit should become a standard sanity check for any new "interpretable" vision model to ensure reported explanations aren't just reflecting leaked context.

Summary updated

Abstract

Part and prototype-based explanations for Vision Transformers (ViTs) present a decision as evidence from individual image regions. However, self-attention mixes information across patches, so part-level features absorb context from the whole image and consequently fail to faithfully reflect the decision. We show that existing interpretable ViT classifiers depend heavily on this leakage: when each explained region is encoded in isolation, their accuracy collapses, and retraining them under isolation recovers only part of the lost accuracy. We propose a classifier that represents each image as a graph of isolated parts. Each part is encoded only from its own patches by a ViT whose last blocks are fine-tuned under isolation, preventing evidence leakage from elsewhere in the image. Parts are discovered without annotation and refined hierarchically through their sub-parts, making them more consistent and stable across images than those of competing methods. Each edge is computed from part pairs, and recovers evidence that attention previously shared across the object without affecting the parts. Matching the image graph to a learned prototype graph per class makes the prediction a direct combination of node and edge similarities, each computed only from the patches it is credited to, so every explanation is faithful by construction. Across 11 fine-grained and general-object benchmarks, our method exceeds the accuracy of existing explainable methods that rely on leakage, while providing explanations that reflect what the model actually uses.
BibTeX
@misc{tomczyk2026keeping,
  author = {Bernard Tomczyk and Gail Collyer-Hoar and Bryan Williams},
  title = {{Keeping Parts Honest: Isolated Part Graphs for Faithful Vision Transformer Explanations}},
  year = {2026},
  eprint = {2611.22853},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22853},
}
Figure