Abstract
Explaining a deep learning model can help users understand its behavior and allow researchers to discern its shortcomings. Recent work has primarily focused on explaining models for tasks like image classification or visual question answering. In this paper, we introduce Salient Attributes for Network Explanation (SANE) to explain image similarity models, where a model's output is a score measuring the similarity of two inputs rather than a classification score. In this task, an explanation depends on both of the input images, so standard methods do not apply. Our SANE explanations pairs a saliency map identifying important image regions with an attribute that best explains the match. We find that our explanations provide additional information not typically captured by saliency maps alone, and can also improve performance on the classic task of attribute recognition. Our approach's ability to generalize is demonstrated on two datasets from diverse domains, Polyvore Outfits and Animals with Attributes 2.
Introduction: The Challenge of Explaining Similarity
Most research in Explainable AI (XAI) has centered on image classification—answering the question, "Why is this image labeled a cat?" In such scenarios, a spatial saliency map (a heatmap highlighting influential pixels) often suffices. However, explaining image similarity models, which are foundational to recommendation systems and image retrieval, presents a unique challenge. Unlike classification, the output depends on the interaction between two distinct inputs. A simple heatmap might indicate that a model is focusing on a specific object, but it fails to convey the semantic reason for the match. For instance, does a necklace match a ring because of its color, material, or brand?
The SANE Framework
To address this, we propose SANE (Salient Attributes for Network Explanation). SANE is a "black-box" framework designed to explain any similarity model regardless of its internal architecture. It provides a dual-modality explanation: a saliency map identifying "where" the model looks and a semantic attribute identifying "what" the model sees.
Methodology
The SANE framework is built upon three core components:
- Saliency Map Generation: We adapt black-box perturbation methods, such as RISE and LIME. By systematically masking portions of the input image and observing the resulting drop in the similarity score, we identify the regions most critical to the model's decision.
- Attribute Prediction with Spatial Alignment: We train an attribute predictor that generates "activation maps" for various semantic labels. A key innovation is using the similarity model's saliency maps as supervision during training, forcing the attribute concepts to align with the regions the similarity model actually utilizes.
- Informative Attribute Selection: To select the most descriptive attribute for a match, SANE ranks candidates based on model confidence, the overlap between attribute activation maps and similarity saliency maps, and a Testing with Concept Activation Vectors (TCAV) score.
Experimental Evaluation
We evaluated SANE across two diverse domains:
- Polyvore Outfits: A fashion dataset containing over 365,000 images, where the task is to determine the compatibility of clothing items.
- Animals with Attributes 2 (AwA2): A natural image dataset used to explain animal similarity based on traits like "claws" or "spots."
Key Results and Insights
- Semantic Clarity: Saliency maps alone are often uninformative when an image contains only one object. SANE's addition of semantic labels (e.g., "golden") provides the necessary context for the "why" behind a match.
- Non-Symmetric Explanations: SANE captures the nuance that reasons for similarity can differ depending on the reference point. A ring might match a necklace because it is "black," while the necklace matches the ring because it is "golden."
- Learning Synergy: Interestingly, supervising attribute activation maps with saliency data improved the model's performance on standard attribute recognition tasks by approximately 3% mAP.
- Human Interpretability: In a user study, participants were significantly better at predicting AI preferences when provided with SANE's explanations. On the AwA2 dataset, SANE attributes improved user accuracy by ~19% over control groups.
Limitations and Future Work
While effective, SANE's reliance on black-box perturbation is computationally intensive, requiring numerous forward passes. Additionally, the system currently depends on a predefined list of attributes. Future research will focus on improving the efficiency of saliency generation and exploring tighter integration with unsupervised attribute discovery to remove the dependency on human-annotated labels.
Summary updated