Interpretable Graph Capsule Networks for Object Recognition

Jindong Gu

Abstract

Capsule Networks (CapsNets) offer a promising alternative to standard Convolutional Neural Networks (CNNs) by modeling hierarchical relationships between object parts. However, traditional CapsNets rely on computationally expensive iterative routing mechanisms that lack transparency. This paper introduces GraCapsNet (Graph Capsule Network), which replaces iterative routing with a multi-head attention-based graph pooling mechanism. GraCapsNet achieves superior performance on complex datasets like CIFAR-10 with significantly fewer parameters while providing inherent interpretability through its attention weights.

Introduction

While CNNs are dominant in image recognition, they are often criticized for their vulnerability to adversarial attacks and their "black-box" nature. Capsule Networks were designed to address these issues by representing objects as a hierarchy of "parts" and "wholes." Despite their potential, the standard "routing by agreement" algorithm is inefficient and difficult to interpret using standard saliency map tools. GraCapsNet aims to modernize this architecture by framing routing as a graph pooling problem, making the model interpretable by design.

Methods: The GraCapsNet Architecture

The fundamental innovation of GraCapsNet is the transition from iterative routing to a graph-based attention mechanism.

Overview of the GraCapsNet architecture featuring multi-head attention-based graph pooling.

The architecture follows these key steps:

  1. Graph Modeling: Low-level features, or primary capsules, are treated as nodes in a graph. Spatial relationships between these parts are explicitly modeled using an adjacency matrix based on Euclidean distance.
  2. Multi-head Attention Graph Pooling: Instead of iterative agreement, the model employs multi-head attention to pool primary capsules into high-level class capsules. This allows different "heads" to specialize in various visual entities.
  3. Built-in Interpretability: The attention weights used during pooling serve as natural saliency maps, indicating which parts of the input image contributed most to the final classification.
  4. Parameter Efficiency: By utilizing shared projections and graph operations, GraCapsNet significantly reduces the number of trainable parameters compared to traditional CapsNets.

Key Results and Discussion

The authors evaluated GraCapsNet against standard benchmarks, including MNIST, Fashion-MNIST, CIFAR-10, and SVHN.

Performance and Efficiency

GraCapsNet demonstrated substantial improvements over the original CapsNet, particularly on more complex data. On the CIFAR-10 dataset, GraCapsNet achieved an accuracy of 82.21%, a significant jump from the original CapsNet's 74.64%. Remarkably, this performance boost was achieved with only 2.9M parameters, compared to the 7.66M required by the original architecture.

Interpretability and Robustness

A core strength of GraCapsNet is its inherent interpretability. Unlike post-hoc explanation methods, GraCapsNet's attention maps are a direct byproduct of its decision-making process.

Visual attention maps generated by GraCapsNet on F-MNIST and CIFAR-10 datasets.

Quantitative evaluations using the Area Over the Perturbation Curve (AOPC) metric confirmed that these built-in explanations are more faithful to the model's behavior than standard methods like Integrated Gradients. Furthermore, the model showed increased resilience to adversarial attacks (e.g., FGSM and C&W) compared to both CNNs and other CapsNet variants.

Conclusion

GraCapsNet represents a significant step forward for capsule-based architectures. By replacing complex iterative routing with efficient graph pooling, the model achieves a rare trifecta: higher accuracy on complex tasks, a smaller parameter footprint, and inherent interpretability. Future work may explore dynamic graph construction and scaling the architecture to high-resolution datasets like ImageNet.

Summary updated

Abstract

Capsule Networks, as alternatives to Convolutional Neural Networks, have been proposed to recognize objects from images. The current literature demonstrates many advantages of CapsNets over CNNs. However, how to create explanations for individual classifications of CapsNets has not been well explored. The widely used saliency methods are mainly proposed for explaining CNN-based classifications; they create saliency map explanations by combining activation values and the corresponding gradients, e.g., Grad-CAM. These saliency methods require a specific architecture of the underlying classifiers and cannot be trivially applied to CapsNets due to the iterative routing mechanism therein. To overcome the lack of interpretability, we can either propose new post-hoc interpretation methods for CapsNets or modifying the model to have build-in explanations. In this work, we explore the latter. Specifically, we propose interpretable Graph Capsule Networks (GraCapsNets), where we replace the routing part with a multi-head attention-based Graph Pooling approach. In the proposed model, individual classification explanations can be created effectively and efficiently. Our model also demonstrates some unexpected benefits, even though it replaces the fundamental part of CapsNets. Our GraCapsNets achieve better classification performance with fewer parameters and better adversarial robustness, when compared to CapsNets. Besides, GraCapsNets also keep other advantages of CapsNets, namely, disentangled representations and affine transformation robustness.
BibTeX
@inproceedings{gu2021interpretable,
  author = {Jindong Gu},
  title = {{Interpretable Graph Capsule Networks for Object Recognition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i2.16237},
}
Figure