Global Interpretability for ProtoPNet Using Rule-Based Explanations
Introduction
Deep learning models, particularly in sensitive domains like medical imaging or biodiversity monitoring, are often criticized for their "black-box" nature. Prototypical Part Networks (ProtoPNets) were developed to mitigate this by providing local explanations: they justify a specific prediction by highlighting image patches that resemble learned "prototypes." For instance, a bird might be classified as a specific species because a patch of its wing matches a known prototype for that species.
However, local explanations do not scale to a global understanding of the model's logic. Auditing a model for spurious correlations or systematic biases requires a global overview, which is currently missing for ProtoPNets. This research introduces a method to extract concise, human-readable IF-THEN rules that summarize the global decision-making process of a ProtoPNet.
Methodology
The proposed framework converts the continuous activation scores of a ProtoPNet into discrete logical rules through a two-stage pipeline.
1. Differentiable Hard Gating
To induce logical rules, the model must first determine the presence or absence of a prototype. The authors implement a differentiable "hard-gating" mechanism. By injecting Gumbel noise into activation scores and applying a temperature-scaled sigmoid function, the network learns to produce binary features (1 if present, 0 if absent) while remaining trainable via backpropagation.
2. Adapted RIPPER Rule Induction
With binarized features, the authors employ a modified version of the RIPPER (Repeated Incremental Pruning to Produce Error Reduction) algorithm. The adaptations include cross-class penalties to discourage rules from using irrelevant prototypes and a "top-N prototype fallback" to ensure the rule set covers the entire dataset effectively.
Experimental Setup
The method was evaluated using a subset of 10 classes from the Caltech Bird Dataset (CUB), a benchmark for fine-grained image classification.
The underlying ProtoPNet utilized a ResNet backbone. For rule extraction, the researchers used a split of 280 test images, with 70% dedicated to training the RIPPER model and 30% held out for final evaluation.
Results and Key Insights
The extracted global rules provided a highly interpretable summary of the model. Across the 10 classes, the algorithm generated only 71 rules, with an average of 2.42 conditions per rule.
| Metric | Value |
|---|---|
| ProtoPNet Accuracy | 85.13% |
| Rule Adherence (to Model) | 70.37% |
| Rule Accuracy (to Ground Truth) | 71.60% |
A significant finding is that the "absence" of a feature is often as informative as its presence. Unlike standard ProtoPNet heatmaps that only show what is there, these rules can explicitly state that a class is identified by the lack of a certain pattern.
Conclusion and Future Work
While there is a performance gap between the full neural network (85.13%) and the rule-based surrogate (71.60%), the extracted rules offer a level of transparency previously unavailable for ProtoPNets. This global view allows domain experts to audit the model's logic efficiently. Future work will focus on closing the adherence gap and incorporating expert knowledge directly into the rule induction process to ensure the model's "logic" aligns with human biological understanding.
Summary updated