Resampling-based Assessment of Robustness to Distribution Shift for Deep Neural Networks

Xudong Sun, Yu Wang, Alexej Gossmann, Bernd Bischl

Introduction

Standard evaluation protocols in machine learning, such as k-fold cross-validation (CV), rely heavily on the assumption that data is independent and identically distributed (i.i.d.). While this provides a consistent measure of performance on seen distributions, it often fails to predict how models will behave in real-world scenarios where distribution shifts are common. This "i.i.d. blind spot" can lead to over-optimistic assessments of a model's reliability.

To address this, the authors propose VGMM-VAE-CV, a novel resampling framework designed to automatically generate "hard" data splits from a single dataset. By simulating semantically meaningful distribution shifts, this method allows researchers to quantify model robustness without the need for manually curated multi-domain datasets.

The VGMM-VAE-CV Framework

The core idea of the proposed method is to identify and isolate "pseudo-subdomains" within a dataset. The process involves three distinct stages:

  1. Latent Representation: A Variational Autoencoder (VAE) is trained to map high-dimensional input data (e.g., images) into a lower-dimensional latent space. This space is designed to capture the underlying generative factors of the data.
  2. Clustering for Shift: Within this latent space, a Variational Gaussian Mixture Model (VGMM) is applied to each class. This groups similar data points together, effectively identifying sub-distributions that might represent different styles, acquisition conditions, or sub-types within a category.
  3. Non-i.i.d. Resampling: Instead of random partitioning, folds are created based on these clusters. By holding out specific clusters during training and using them for testing, the framework forces the model to generalize to unseen subdomains.
Latent space clustering of Fashion-MNIST using VGMM to identify pseudo-subdomains.

To validate the severity of the generated shifts, the authors utilize the Wasserstein distance (Earth Mover's Distance) to measure the divergence between the training and test distributions.

Experimental Evaluation

The framework was evaluated using the Fashion-MNIST dataset, consisting of 70,000 grayscale images across 10 clothing categories. The authors tested several standard deep learning architectures, including AlexNet, LeNet, and a custom 3conv3fc model. Notably, they also included Bayesian Convolutional Neural Networks (BCNNs) to test the common hypothesis that Bayesian methods offer superior robustness to distribution shift.

Key Findings

The results demonstrate a dramatic performance gap between standard CV and the proposed VGMM-VAE-CV framework.

1. Significant Accuracy Degradation

Across all architectures, moving from an i.i.d. validation set to a shifted test set resulted in an average accuracy drop of 26.0 percentage points. This highlights that models which appear highly accurate under standard CV may fail significantly when encountering subtle semantic shifts.

2. Increased Performance Variability

The variance in test accuracy across different folds was 14 times higher under the VGMM-VAE-CV scheme compared to standard CV. This indicates that model reliability becomes highly unpredictable when the test distribution deviates from the training distribution.

Performance comparison of AlexNet under standard CV vs. the proposed VGMM-CV, showing significant test accuracy degradation and high variance under distribution shift.

3. The Bayesian Paradox

A critical insight from the study is that Bayesian CNNs, despite their ability to model weight uncertainty, are just as vulnerable to distribution shift as frequentist models. While BCNNs showed less overfitting on training data, their performance on the shifted test sets mirrored the steep declines seen in standard models.

Conclusion

The VGMM-VAE-CV framework provides a scalable and automated way to assess model robustness. By exposing the fragility of deep learning models to non-i.i.d. data, it serves as a more rigorous benchmark for real-world deployment. Future work aims to provide more explicit control over the severity of the generated shifts and to extend the framework to non-image domains such as natural language processing.

Summary updated

Abstract

A novel variational inference based resampling framework is proposed to evaluate the robustness and generalization capability of deep learning models with respect to distribution shift. We use Auto Encoding Variational Bayes to find a latent representation of the data, on which a Variational Gaussian Mixture Model is applied to deliberately create distribution shift by dividing the dataset into different clusters. Wasserstein distance is used to characterize the extent of distribution shift between the generated data splits. We compare several popular Convolutional Neural Network (CNN) architectures and Bayesian CNN models for image classification on the Fashion-MNIST dataset, to assess their robustness and generalization behavior under the deliberately created distribution shift, as well as under random Cross Validation. Our method of creating artificial domain splits of a single dataset can also be used to establish novel model selection criteria and assessment tools in machine learning, as well as benchmark methods for domain adaptation and domain generalization approaches.
BibTeX
@inproceedings{sun2019resampling,
  author = {Xudong Sun and Yu Wang and Alexej Gossmann and Bernd Bischl},
  title = {{Resampling-based Assessment of Robustness to Distribution Shift for Deep Neural Networks}},
  booktitle = {SSCI},
  year = {2019},
  url = {https://openreview.net/forum?id=kGiQufjPYM},
}
Figure