Abstract
Conventionally, AI models are thought to trade off explainability for lower accuracy. We develop a training strategy that not only leads to a more explainable AI system for object classification, but as a consequence, suffers no perceptible accuracy degradation. Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. This is represented in the form of a saliency map conveying how much each pixel contributed to the network's decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the image regions that directly correspond to the ground-truth object. We quantify explainability using an automated metric, and using human judgement. We propose explainability as a means for bridging the visual-semantic gap between different domains where model explanations are used as a means of disentagling domain specific information from otherwise relevant features. We demonstrate that this leads to improved generalization to new domains without hindering performance on the original domain.
Introduction
Deep learning models often rely on "spurious correlations" or dataset bias to achieve high accuracy. For instance, a model might identify a "cow" primarily by the presence of green grass. While effective within a specific training set, this strategy fails in Domain Generalization (DG), where the model must perform on unseen "target" domains like sketches or paintings. This research addresses the challenging single-source DG setting by leveraging explainability to bridge the visual-semantic gap.
Methodology: XAI Training Strategy
The core innovation is an XAI-driven training loop that periodically "nudges" the model's internal attention to ensure it focuses on class-relevant features rather than background noise.
- Saliency Mapping: The system uses Grad-CAM to generate saliency maps, identifying the pixels most influential to a classification decision.
- Explainability Feedback: Every few epochs, the model's peak saliency is compared against a ground-truth object mask. If the model is focusing on the background, a binary mask is applied to internal activations (typically at the final spatial layer of a ResNet-50), forcing the network to suppress irrelevant features and re-focus on the object.
- Inference Efficiency: This feedback mechanism is only active during training. At test time, the model remains a standard, efficient CNN.
Results and Discussion
The XAI training strategy achieved a 2.56% relative accuracy improvement across diverse target domains (including Clipart, Infographs, and Quickdraw) compared to standard baselines. Crucially, this improvement came with no "accuracy tax" on the original source domain.
Beyond quantitative metrics, the model demonstrated superior alignment with human reasoning. In crowdsourced evaluations, humans preferred the XAI model's explanations in 80% of cases. This suggests that by forcing the model to "look" at the right parts of an image, we successfully disentangle core content from domain-specific context.
Conclusion
This work demonstrates that explainability is not just a post-hoc diagnostic tool but a powerful regularizer for training more robust models. By enforcing human-like attention during training, models become more interpretable and generalize better to novel environments. Future work will focus on reducing the dependency on expensive segmentation masks and extending the strategy to multi-label and multi-object scenarios.
Summary updated