Analyzing Effects of Mixed Sample Data Augmentation on Model Interpretability

Soyoun Won, Sung-Ho Bae, Seong Tae Kim

Introduction

Mixed Sample Data Augmentation (MSDA) techniques, such as Mixup and CutMix, have become standard tools for improving the accuracy and generalization of deep neural networks. By blending multiple images and their corresponding labels during training, these methods force models to learn more robust features. However, this paper identifies a significant hidden cost: a degradation in model interpretability. The authors argue that while MSDA models appear more robust, their internal decision-making processes become less transparent and their explanations less faithful to the underlying logic. A primary challenge in identifying this trend has been the bias in existing interpretability metrics, which often conflate a model's inherent robustness to missing data with the quality of its explanations.

The Robustness Bias and Inter-Model Deletion

Traditional "deletion" metrics measure interpretability by observing how quickly a model's confidence drops as important pixels (identified by an attribution map) are removed. The authors demonstrate that this approach is flawed when comparing different models. Because MSDA models are specifically trained to be robust to occlusions, their confidence naturally drops more slowly even when the attribution map is poor. This creates a "robustness trap" where MSDA models falsely appear more interpretable.

To address this, the authors introduce Inter-Model Deletion. This novel metric isolates the true faithfulness of an explanation by calculating the difference between a "Least Relevant First" (LeRF) deletion curve and a "Random Order" (RaO) deletion curve. By subtracting the random baseline, the metric cancels out the model's inherent robustness to random pixel loss, providing a fair comparison of how well the attribution method captures the model's actual reasoning.

Methodology and Localization

The study evaluates interpretability through two primary dimensions: localization and faithfulness. For localization, the authors move beyond simple bounding box overlaps to more nuanced measures: Energy-based Pointing Game (EnergyPG) and Effective Hit Ratio (EHR). These metrics assess the intensity or "energy" of the attribution map within the object's boundaries, offering a more precise evaluation of whether the model is focusing on the correct visual features.

Experimental Results

Experiments conducted on ImageNet and CIFAR-10 using architectures like ResNet-50 and Vision Transformers (ViT) reveal a consistent decline in interpretability for MSDA-trained models.

Comparison of attribution maps and localization metrics across various MSDA techniques. Baseline models consistently show more concentrated energy on the target object compared to MSDA variants.

As shown in the quantitative results, the baseline ResNet-50 model achieved an EnergyPG score of 0.559 on ImageNet, significantly higher than Mixup (0.489) and CutMix (0.500). The faithfulness gap is even more pronounced: on CIFAR-10, the baseline model's Inter-Model Deletion score was 27.53, while Mixup's score fell to 12.15. This indicates that explanations for Mixup models are much less representative of the model's internal logic.

The Role of Label Mixing

Through ablation studies dissecting Mixup into Image-Mixing-Only and Label-Mixing-Only components, the authors pinpoint label mixing as the primary driver of degraded interpretability. Assigning soft, blended labels (e.g., 0.5/0.5 for a mix of two classes) weakens the clear correspondence between specific visual features and the model's output. This leads to "messier" attribution maps that often cover broad silhouettes and include irrelevant background information.

Qualitative examples of attribution maps from MSDA models, showing a tendency toward broad silhouettes and the inclusion of non-relevant background pixels.

Conclusion and Future Directions

This research highlights a critical trade-off between accuracy and interpretability in modern data augmentation. While MSDA improves performance, it complicates the task of understanding why a model makes a certain prediction. The authors suggest that the proposed Inter-Model Deletion metric could serve as a loss function during training to help recover lost interpretability. Future work is needed to extend these findings to other domains like NLP and to conduct human-subject studies to determine the practical impact of these degraded explanations on end-users.

Summary updated

Abstract

Data augmentation strategies are actively used when training deep neural networks (DNNs). Recent studies suggest that they are effective at various tasks. However, the effect of data augmentation on DNNs' interpretability is not yet widely investigated. In this paper, we explore the relationship between interpretability and data augmentation strategy in which models are trained with different data augmentation methods and are evaluated in terms of interpretability. To quantify the interpretability, we devise three evaluation methods based on alignment with humans, faithfulness to the model, and the number of human-recognizable concepts in the model. Comprehensive experiments show that models trained with mixed sample data augmentation show lower interpretability, especially for CutMix and SaliencyMix augmentations. This new finding suggests that it is important to carefully adopt mixed sample data augmentation due to the impact on model interpretability, especially in mission-critical applications.
BibTeX
@article{won2025analyzing,
  author = {Soyoun Won and Sung-Ho Bae and Seong Tae Kim},
  title = {{Analyzing Effects of Mixed Sample Data Augmentation on Model Interpretability}},
  journal = {Neural Networks},
  year = {2025},
  doi = {10.1016/j.neunet.2025.107611},
}
Figure