Explainable Deep Classification Models for Domain Generalization

Andrea Zunino, Sarah Adel Bargal, Riccardo Volpi, Mehrnoosh Sameki, Jianming Zhang, Stan Sclaroff, Vittorio Murino, Kate Saenko

Abstract

Conventionally, AI models are thought to trade off explainability for lower accuracy. We develop a training strategy that not only leads to a more explainable AI system for object classification, but as a consequence, suffers no perceptible accuracy degradation. Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. This is represented in the form of a saliency map conveying how much each pixel contributed to the network's decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the image regions that directly correspond to the ground-truth object. We quantify explainability using an automated metric, and using human judgement. We propose explainability as a means for bridging the visual-semantic gap between different domains where model explanations are used as a means of disentagling domain specific information from otherwise relevant features. We demonstrate that this leads to improved generalization to new domains without hindering performance on the original domain.

Introduction

Deep learning models often rely on "spurious correlations" or dataset bias to achieve high accuracy. For instance, a model might identify a "cow" primarily by the presence of green grass. While effective within a specific training set, this strategy fails in Domain Generalization (DG), where the model must perform on unseen "target" domains like sketches or paintings. This research addresses the challenging single-source DG setting by leveraging explainability to bridge the visual-semantic gap.

Overview of domain generalization across diverse styles and the impact of XAI feedback on saliency maps.

Methodology: XAI Training Strategy

The core innovation is an XAI-driven training loop that periodically "nudges" the model's internal attention to ensure it focuses on class-relevant features rather than background noise.

  1. Saliency Mapping: The system uses Grad-CAM to generate saliency maps, identifying the pixels most influential to a classification decision.
  2. Explainability Feedback: Every few epochs, the model's peak saliency is compared against a ground-truth object mask. If the model is focusing on the background, a binary mask is applied to internal activations (typically at the final spatial layer of a ResNet-50), forcing the network to suppress irrelevant features and re-focus on the object.
  3. Inference Efficiency: This feedback mechanism is only active during training. At test time, the model remains a standard, efficient CNN.

Results and Discussion

The XAI training strategy achieved a 2.56% relative accuracy improvement across diverse target domains (including Clipart, Infographs, and Quickdraw) compared to standard baselines. Crucially, this improvement came with no "accuracy tax" on the original source domain.

Visual evidence comparison between graphics and real-world domains, showing how the model focuses on object-specific features.

Beyond quantitative metrics, the model demonstrated superior alignment with human reasoning. In crowdsourced evaluations, humans preferred the XAI model's explanations in 80% of cases. This suggests that by forcing the model to "look" at the right parts of an image, we successfully disentangle core content from domain-specific context.

Conclusion

This work demonstrates that explainability is not just a post-hoc diagnostic tool but a powerful regularizer for training more robust models. By enforcing human-like attention during training, models become more interpretable and generalize better to novel environments. Future work will focus on reducing the dependency on expensive segmentation masks and extending the strategy to multi-label and multi-object scenarios.

Summary updated

Abstract

Conventionally, AI models are thought to trade off explainability for lower accuracy. We develop a training strategy that not only leads to a more explainable AI system for object classification, but as a consequence, suffers no perceptible accuracy degradation. Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. This is represented in the form of a saliency map conveying how much each pixel contributed to the network's decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the image regions that directly correspond to the ground-truth object. We quantify explainability using an automated metric, and using human judgement. We propose explainability as a means for bridging the visual-semantic gap between different domains where model explanations are used as a means of disentagling domain specific information from otherwise relevant features. We demonstrate that this leads to improved generalization to new domains without hindering performance on the original domain.
BibTeX
@inproceedings{zunino2021explainable,
  author = {Andrea Zunino and Sarah Adel Bargal and Riccardo Volpi and Mehrnoosh Sameki and Jianming Zhang and Stan Sclaroff and Vittorio Murino and Kate Saenko},
  title = {{Explainable Deep Classification Models for Domain Generalization}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  url = {https://openreview.net/forum?id=m4mRS8SzOe},
}
Figure