Pantypes: Diverse Representatives for Self-Explainable Models

Rune D. Kjærsgaard, Ahcène Boubekki, Line H. ClemmensenAAAI 2024 · DOI

TL;DR

This paper introduces "pantypes," a novel approach to creating more diverse and representative prototypes in self-explainable AI (SEAI) models. By using a new volumetric loss function, pantypes effectively capture the full spectrum of data diversity, leading to improved interpretability and fairness without sacrificing predictive accuracy.

What Problem Does This Paper Address?

The field of Explainable AI (XAI) aims to make machine learning (ML) models more transparent and trustworthy. Self-explainable models (SEMs) are a promising avenue, often relying on learned "prototypes" to represent classes and justify decisions. While existing SEMs focus on transparency and trustworthiness, they often struggle with diversity. Specifically, learned prototypes can fail to adequately represent all aspects of the data distribution, especially in low-density regions. This "representation bias" can lead to unfairness and detrimental properties in ML systems. The paper addresses this gap by proposing a method to generate prototypes that better capture the full diversity of the data.

Methods

The authors build upon the ProtoVAE (Prototypes as Variational Autoencoders) framework. Their core innovation is the introduction of a volumetric diversity loss (Lvol) that replaces the standard orthonormality loss used in ProtoVAE. This new loss is inspired by Determinantal Point Processes (DPPs), which are known for their ability to sample diverse sets.

The volumetric loss encourages prototypes to occupy distinct regions of the latent space, thereby increasing their diversity and coverage of the data distribution. A key feature of this loss is its ability to dynamically determine the number of prototypes needed for each class. For complex classes, more pantypes are learned, while simpler classes are represented by a sparser set. This "prototype pruning" mechanism enhances interpretability.

The overall model, named PanVAE, uses the following loss structure: LPanVAE = Lpred + LVAE + Lvol.

Datasets

The following datasets were used for training and evaluation:

  • MNIST: A dataset of handwritten digits.
  • FashionMNIST (FMNIST): A dataset of clothing images.
  • QuickDraw (QDraw): A dataset of sketches.
  • CelebA: A dataset of celebrity faces.
  • UTK Face: A dataset of facial images used to evaluate demographic diversity and bias.

Key Insights and Ideas

  • Representation Bias is a Key Issue: The paper highlights that even with diversity-promoting losses like orthonormality, SEMs can suffer from representation bias if prototypes cluster in small regions of the latent space.
  • Volumetric Loss for True Diversity: The proposed volumetric loss, inspired by DPPs, directly encourages prototypes to spread out and cover larger volumes of the latent space, leading to more comprehensive data representation.
  • Dynamic Prototype Selection: The volumetric loss naturally leads to a dynamic selection of prototypes, allowing for more complex classes to have more representatives while keeping simpler classes interpretable with fewer.
  • Geometric vs. Combinatorial Diversity: The paper distinguishes between "geometric diversity" (visual spread of prototypes) and "combinatorial diversity" (fair representation of discrete attributes like race). The volumetric loss primarily enhances geometric diversity, which can indirectly help with combinatorial diversity, especially when data is pre-processed to reduce confounding factors.
  • Quantitative Measures for Diversity: The authors advocate for quantitative measures (like the Davies-Bouldin index and data coverage) to assess prototype diversity, moving beyond purely qualitative visual inspections.

Key Results

  • Predictive Performance: PanVAE achieves comparable predictive performance (accuracy) to ProtoVAE across all tested datasets, demonstrating that increased diversity does not come at the cost of accuracy. For example, on MNIST, PanVAE achieved 99.4% accuracy, very close to ProtoVAE's 99.3%.
  • Improved Prototype Representation: PanVAE consistently achieved lower Davies-Bouldin (DB) index scores across all datasets, indicating better quality and representativity of its prototypes compared to ProtoVAE and ProtoPNet. For instance, on FMNIST, PanVAE had a DB score of 1.1 ± 0.0, significantly lower than ProtoVAE's 1.4 ± 0.0.
  • Enhanced Data Coverage: Qualitative and quantitative evaluations (e.g., Figure 4) show that PanVAE pantypes cover a significantly larger portion of the latent space. For the "bag" class in FMNIST, PanVAE pantypes covered 77% of the data volume, compared to ProtoVAE's 33%.
  • Mitigation of Demographic Bias: On the UTK Face dataset, PanVAE showed a lower accuracy gap between White males and Black females compared to ProtoVAE, suggesting its improved diversity can help mitigate demographic bias. However, this came with a slight decrease in accuracy for the majority group (White males).

Limitations and Future Work

  • Geometric vs. Combinatorial Diversity Alignment: The paper notes that while the volumetric loss promotes geometric diversity, this doesn't automatically guarantee alignment with combinatorial diversity (e.g., demographic fairness). This misalignment can occur if visual variations (like pose or background) are more pronounced than demographic variations.
  • Data Pre-processing for Fairness: To enforce high demographic diversity, images might need pre-processing (e.g., alignment, background removal) or sensitive features might need to be directly incorporated into the model.
  • Trade-off in Majority Group Performance: Achieving better fairness for minority groups might come at the expense of slightly reduced performance for the majority group. The choice of representation needs careful consideration based on the model's aim and target population.
  • Future Work: The authors suggest that future work should focus on how to enforce high demographic diversity more directly and explore the trade-offs involved. They also emphasize the importance of quantitative evaluation of diversity alongside trustworthiness.

Summary updated

BibTeX
@inproceedings{kjrsgaard2024pantypes,
  author = {Rune D. Kjærsgaard and Ahcène Boubekki and Line H. Clemmensen},
  title = {{Pantypes: Diverse Representatives for Self-Explainable Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i12.29223},
}
Figure