Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks

Nicola Garau, Niccolò Bisagno, Zeno Sambugaro, Nicola ConciCVPR 2022 · DOI · arXiv

Deep learning has a transparency problem. While models like ResNet or Vision Transformers (ViT) achieve state-of-the-art accuracy, they often operate as "black boxes," making it difficult for humans to understand how they move from raw pixels to a final classification. In their research, Nicola Garau and colleagues from the University of Trento introduce Agglomerator, a novel architecture that moves away from standard feed-forward processing toward a biologically inspired structure that explicitly models how parts (like wheels or ears) form wholes (like cars or dogs).

The Agglomerator Framework

Agglomerator is built on the concept of hyper-columns found in the human neocortex. Unlike standard Convolutional Neural Networks (CNNs) that rely on pooling operations—which often discard spatial relationships—Agglomerator preserves structural information through a multi-stage process:

  1. Patch-based Embedding: The input image is broken into patches, each fed into a dedicated "column."
  2. Stacked Auto-encoders: Each column consists of several levels ($K$) connected by top-down and bottom-up auto-encoders that share weights across columns.
  3. Iterative Routing (The Propagation Phase): Instead of a single pass, the model iteratively routes information. Neighboring columns communicate to reach a consensus. At lower levels, they might agree on a "part" (e.g., an eye); at the top level, they agree on the "whole" (e.g., a dog).
Overview of the Agglomerator architecture, illustrating the pipeline from patch-based embedding to the emergence of part-whole hierarchies and semantic clustering.

Interpretability through "Islands of Agreement"

A core innovation of the Agglomerator is its inherent interpretability. During the propagation phase, groups of columns representing the same object begin to output similar values, forming what the authors call "islands of agreement." These islands can be visually mapped to see exactly which parts of the image the network identifies as belonging to an object versus the background. This provides a direct window into the model's internal logic, showing the intermediate steps where textures are recognized as parts, and parts are synthesized into wholes.

Semantic Latent Space and Results

Agglomerator doesn't just separate classes; it organizes its internal knowledge into a Semantic Latent Space. In 2D visualizations, broad categories like "Animals" and "Vehicles" form distinct clusters. Interestingly, categories like "birds" and "airplanes" are positioned near the boundary between these super-classes because they share the conceptual feature of "flying," mimicking human-like lexical hierarchies.

The model was evaluated on benchmarks including MNIST, FashionMNIST, and CIFAR-10/100. Key results include:

  • Superior Scaling: On CIFAR-100, Agglomerator achieved a 40.97% error rate, significantly outperforming traditional Capsule Networks (82.00%).
  • Efficiency: The model uses approximately 72 million parameters, making it far more compact than heavyweights like ViT-L/16 (632 million) while remaining competitive.
  • Reduced Overlap: The authors measured "Overlap (O)," representing the probability of high-level hierarchical mistakes. Agglomerator achieved an O=2%, vastly superior to ResNet-110 (12%) and ViT (24%).
Comparison of latent space organization across different models. Agglomerator (Ours) achieves the lowest overlap (O=2%) between semantic clusters, indicating a more structured internal representation.

Conclusion and Future Directions

Agglomerator demonstrates that neural networks can be both high-performing and interpretable by adopting biologically inspired architectural constraints. While the model currently requires careful hyper-parameter tuning and robust pre-training, it offers a promising path toward more transparent AI. Future work will focus on scaling the architecture to larger datasets like ImageNet and refining pre-training strategies for deployment in safety-critical fields such as healthcare and autonomous driving.

Summary updated

BibTeX
@inproceedings{garau2022interpretable,
  author = {Nicola Garau and Niccolò Bisagno and Zeno Sambugaro and Nicola Conci},
  title = {{Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.01332},
}
Figure