Is ProtoPNet Really Explainable? Evaluating and Improving the Interpretability of Prototypes

Qihan Huang, Mengqi Xue, Haofei Zhang, Jie Song, Mingli SongarXiv 2022 · OpenReview

Introduction

Part-prototype networks, such as ProtoPNet, are designed to be "self-explainable" by identifying specific object parts (prototypes) to justify classification decisions. For instance, a model might classify an image as a bird because it identifies a region resembling a previously learned "wing" prototype. However, this paper identifies a significant "semantic gap": while prototypes may be close in a high-dimensional feature space, they often fail to correspond to consistent physical parts in real-world images.

The Semantic Gap: Inconsistency and Instability

The authors identify two primary failures in existing prototype-based models:

  1. Inconsistency: A single prototype may highlight a head in one image but a tail in another, undermining the semantic meaning of the "explanation."
  2. Instability: Small amounts of noise or slight image perturbations can cause the model's focus to shift entirely to a different object part.
Visualizing the core failures of part-prototype networks: (a) Inconsistency across different images and (b) Instability under image perturbation.

To address these issues, the researchers developed the first quantitative framework for evaluating interpretability, introducing Consistency and Stability scores based on ground-truth part annotations.

Proposed Methodology: EvalProtoPNet

The paper proposes two architectural improvements to bridge the semantic gap:

Shallow-Deep Feature Alignment (SDFA)

Deep layers in convolutional neural networks (CNNs) excel at capturing semantic information but often lose spatial precision. The SDFA module forces deep feature maps to align their spatial similarity structure with shallow layers, which retain more raw geometric information. This ensures that prototypes remain anchored to specific physical locations.

Score Aggregation (SA)

Standard ProtoPNets often assign negative weights to prototypes, creating a "paradoxical" situation where identifying a relevant part actually decreases the probability of the correct class. The SA module replaces the standard fully-connected layer with a restricted aggregation mechanism, ensuring that prototypes only contribute positively to their assigned categories.

The proposed EvalProtoPNet architecture, featuring the Shallow-Deep Feature Alignment (SDFA) module and the Score Aggregation (SA) module.

Experimental Results

The proposed EvalProtoPNet was evaluated on fine-grained benchmarks including CUB-200-2011, Stanford Cars, and PartImageNet. The results demonstrate that interpretability and accuracy are not a trade-off:

  • Interpretability Gains: On the CUB-200-2011 dataset using a ResNet34 backbone, the Consistency Score improved from 15.1% in the vanilla ProtoPNet to 70.6%.
  • Accuracy Improvements: EvalProtoPNet achieved state-of-the-art accuracy (up to 88.5%), proving that more consistent and stable prototypes lead to better classification performance.
  • Robustness: The model maintained high stability even under adversarial attacks, where previous models' explanations typically collapsed.

Conclusion

This work demonstrates that the "explanations" provided by early part-prototype networks were often unreliable. By introducing quantitative metrics and structural improvements like SDFA and SA, the authors show that it is possible to build models that are both more interpretable and more accurate. Future work will focus on scaling these methods to complex, multi-object scenes and domains where part annotations are scarce.

Summary updated

BibTeX
@misc{huang2022is,
  author = {Qihan Huang and Mengqi Xue and Haofei Zhang and Jie Song and Mingli Song},
  title = {{Is ProtoPNet Really Explainable? Evaluating and Improving the Interpretability of Prototypes}},
  year = {2022},
  url = {https://openreview.net/forum?id=ByNQs1BZRII},
}
Figure