Global Interpretability for ProtoPNet Using Rule-Based Explanations

Alec Parise, Brian Mac NameexAI (Late-breaking Work, Demos, Doctoral Consortium) 2025

Global Interpretability for ProtoPNet Using Rule-Based Explanations

Introduction

Deep learning models, particularly in sensitive domains like medical imaging or biodiversity monitoring, are often criticized for their "black-box" nature. Prototypical Part Networks (ProtoPNets) were developed to mitigate this by providing local explanations: they justify a specific prediction by highlighting image patches that resemble learned "prototypes." For instance, a bird might be classified as a specific species because a patch of its wing matches a known prototype for that species.

However, local explanations do not scale to a global understanding of the model's logic. Auditing a model for spurious correlations or systematic biases requires a global overview, which is currently missing for ProtoPNets. This research introduces a method to extract concise, human-readable IF-THEN rules that summarize the global decision-making process of a ProtoPNet.

Methodology

The proposed framework converts the continuous activation scores of a ProtoPNet into discrete logical rules through a two-stage pipeline.

1. Differentiable Hard Gating

To induce logical rules, the model must first determine the presence or absence of a prototype. The authors implement a differentiable "hard-gating" mechanism. By injecting Gumbel noise into activation scores and applying a temperature-scaled sigmoid function, the network learns to produce binary features (1 if present, 0 if absent) while remaining trainable via backpropagation.

2. Adapted RIPPER Rule Induction

With binarized features, the authors employ a modified version of the RIPPER (Repeated Incremental Pruning to Produce Error Reduction) algorithm. The adaptations include cross-class penalties to discourage rules from using irrelevant prototypes and a "top-N prototype fallback" to ensure the rule set covers the entire dataset effectively.

Experimental Setup

The method was evaluated using a subset of 10 classes from the Caltech Bird Dataset (CUB), a benchmark for fine-grained image classification.

A sample image from the CUB dataset used for evaluating the rule-based ProtoPNet.

The underlying ProtoPNet utilized a ResNet backbone. For rule extraction, the researchers used a split of 280 test images, with 70% dedicated to training the RIPPER model and 30% held out for final evaluation.

Results and Key Insights

The extracted global rules provided a highly interpretable summary of the model. Across the 10 classes, the algorithm generated only 71 rules, with an average of 2.42 conditions per rule.

Metric Value
ProtoPNet Accuracy 85.13%
Rule Adherence (to Model) 70.37%
Rule Accuracy (to Ground Truth) 71.60%

A significant finding is that the "absence" of a feature is often as informative as its presence. Unlike standard ProtoPNet heatmaps that only show what is there, these rules can explicitly state that a class is identified by the lack of a certain pattern.

An example of negative reasoning in the extracted rules, where the absence of a specific spotted prototype helps define the class boundary.

Conclusion and Future Work

While there is a performance gap between the full neural network (85.13%) and the rule-based surrogate (71.60%), the extracted rules offer a level of transparency previously unavailable for ProtoPNets. This global view allows domain experts to audit the model's logic efficiently. Future work will focus on closing the adherence gap and incorporating expert knowledge directly into the rule induction process to ensure the model's "logic" aligns with human biological understanding.

Summary updated

BibTeX
@inproceedings{parise2025global,
  author = {Alec Parise and Brian Mac Namee},
  title = {{Global Interpretability for ProtoPNet Using Rule-Based Explanations}},
  booktitle = {xAI (Late-breaking Work, Demos, Doctoral Consortium)},
  year = {2025},
}
Figure