Interpretable Image Recognition by Constructing Transparent Embedding Space

Jiaqi Wang, Huafeng Liu, Xinyue Wang, Liping Jing

Introduction: The Black Box Problem in CNNs

Deep convolutional neural networks (CNNs) have achieved remarkable accuracy in image recognition, yet they often operate as "black boxes." While we can observe their final classifications, the internal logic—the specific visual features and their combinations that lead to a decision—remains opaque. This lack of interpretability is particularly problematic in fine-grained recognition tasks, such as distinguishing between similar bird species or car models, where "concept entanglement" often occurs. In these cases, a model might conflate unrelated features, making its explanations messy and unhelpful for human auditing.

TesNet: A Transparent Embedding Space

To address these challenges, the authors propose TesNet (Transparent Embedding Space Network). TesNet is designed as a "plug-in" transparency layer that can be integrated into standard CNN architectures like VGG, ResNet, or DenseNet. Its primary goal is to replace high-level, abstract feature vectors with a structured and transparent embedding space.

The core of TesNet's innovation lies in its mathematical constraints:

  • Grassmann Manifolds: Features are constrained to live on a Grassmann manifold, where each category is assigned its own linear subspace.
  • Orthonormality and Disentanglement: Within each category's subspace, the model learns "basis concepts" (e.g., "beak," "wing," "tail") that are forced to be mathematically orthogonal. This ensures that learned concepts are diverse and do not overlap, effectively solving the problem of concept entanglement.
  • Patch-Based Transparency: These abstract basis concepts are mapped back to representative image patches from the training set. When processing a new image, the model identifies which patches most closely resemble these learned concepts.

The "Scoring Sheet" for Human Auditing

One of the most significant advantages of TesNet is its ability to generate a human-readable "scoring sheet" for its decisions. Instead of a single probability score, the model provides a weighted sum of evidence. It can explicitly show which parts of an image contributed to a classification and, crucially, why it rejected alternative classifications.

TesNet provides a transparent "scoring sheet" by comparing image patches to learned category-specific concepts, showing both positive evidence for the correct class and negative evidence against others.

As shown in the figure above, the model identifies specific regions (like the head or wing) and compares them to prototypical patches of a "Yellow-headed Blackbird." Simultaneously, it demonstrates why the bird is not a "Cape Glossy Starling" by highlighting the lack of similarity in key areas.

Performance and Key Results

A common concern with interpretable models is that transparency comes at the cost of accuracy. However, TesNet demonstrates that interpretability can actually serve as a powerful form of regularization. By forcing the model to focus on distinct, repeatable features and ignore background noise, TesNet often outperforms its non-interpretable counterparts.

  • CUB-200-2011 (Birds): Using a DenseNet-121 backbone, TesNet achieved 84.8% accuracy, surpassing the non-interpretable baseline (81.8%) and previous interpretable methods like ProtoPNet (79.0%).
  • Stanford Cars: The model reached 93.1% accuracy, proving its effectiveness on rigid, man-made objects where specific part identification (grilles, headlights) is critical.

Conclusion and Future Directions

TesNet represents a significant step toward making deep learning models more accountable and understandable. By structuring the embedding space through Grassmann manifolds and orthogonality constraints, it achieves high-fidelity interpretability without sacrificing—and often improving—predictive performance. Future work may involve incorporating hierarchical concept structures (e.g., taxonomic relationships) and exploring human-in-the-loop systems where experts can refine the machine-discovered concepts to better align with domain knowledge.

Summary updated

Abstract

Humans usually explain their reasoning (e.g. classification) by dissecting the image and pointing out the evidence from these parts to the concepts in their minds. Inspired by this cognitive process, several part-level interpretable neural network architectures have been proposed to explain the predictions. However, they suffer from the complex data structure and confusing the effect of the individual part to output category. In this work, an interpretable image recognition deep network is designed by introducing a plug-in transparent embedding space (TesNet) to bridge the high-level input patches (e.g. CNN feature maps) and the out- put categories. This plug-in embedding space is spanned by transparent basis concepts which are constructed on the Grassmann manifold. These basis concepts are enforced to be category-aware and within-category concepts are orthogonal to each other, which makes sure the embedding space is disentangled. Meanwhile, each basis concept can be traced back to the particular image patches, thus they are transparent and friendly to explain the reasoning process. By comparing with state-of-the-art interpretable methods, TesNet is much more beneficial to classification tasks, esp. providing better interpretability on predictions and improve the final accuracy. The code is available at https://github.com/JackeyWang96/TesNet.
BibTeX
@inproceedings{wang2021interpretable,
  author = {Jiaqi Wang and Huafeng Liu and Xinyue Wang and Liping Jing},
  title = {{Interpretable Image Recognition by Constructing Transparent Embedding Space}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00093},
}
Figure