Abstract
For many real-world applications, obtaining stable and robust statistical performance is more important than simply achieving state-of-the-art predictive test accuracy. Data augmentation schemes have been shown to improve robustness with respect to input perturbations and domain shifts. Motivated by this, we introduce NoisyMix, a novel training scheme that promotes stability and leverages noisy augmentations in input and feature space to improve both model robustness and in-domain accuracy. NoisyMix produces models that are consistently more robust and provide well-calibrated estimates of class membership probabilities. We demonstrate the benefits of NoisyMix on a range of benchmark datasets, including ImageNet-C, ImageNet-R, and ImageNet-P.
Introduction: The Brittleness of Deep Learning
While modern neural networks achieve impressive accuracy on clean, curated datasets, they remain notoriously "brittle" when deployed in the real world. A standard ResNet-50 might perform well on ImageNet, but its accuracy can drop by over 35% when faced with common corruptions like fog, motion blur, or digital noise.
This research addresses input stability—the sensitivity of a model to data perturbations. Interestingly, standard adversarial training, which targets high-frequency malicious attacks, often degrades a model's ability to handle low-frequency "natural" corruptions. NoisyMix is designed to bridge this gap, providing a training framework that handles real-world "messiness" without sacrificing clean-set performance.
The NoisyMix Framework
NoisyMix is a holistic training scheme built on three pillars:
- Diverse Data Augmentations: Building upon the "AugMix" approach, NoisyMix utilizes multiple augmentation chains (e.g., rotation, shearing, color-shifting) to create a diverse set of transformed images.
- Noisy Feature Mixup (NFM): Beyond input-level mixing, NoisyMix injects noise and performs linear interpolations (Mixup) within the hidden feature spaces. This encourages the model to learn smoother decision boundaries and acts as a powerful implicit regularizer.
- Stability Training via JSD: The framework employs Jensen-Shannon Divergence (JSD) as a consistency loss. This ensures the model's predicted probability distribution for a clean image remains consistent across its various augmented and noisy versions.
Mathematically, minimizing the NoisyMix loss is equivalent to minimizing an upper bound on a combination of adversarial loss and data-dependent regularization, providing a theoretical foundation for its empirical success.
Experimental Evaluation
The authors evaluated NoisyMix against several state-of-the-art methods across a suite of robustness benchmarks including ImageNet-C (synthetic corruptions), ImageNet-R (natural domain shifts), and ImageNet-P (temporal stability).
Key Results
NoisyMix consistently outperformed existing methods across different architectures. On ImageNet-C, it improved robust accuracy by 13% over the baseline and 4% over AugMix. It also significantly improved temporal stability, reducing the "Mean Flip Rate" on ImageNet-P from 58% to 28.5%.
The table below details the performance gains across the ImageNet benchmark suite, highlighting NoisyMix's ability to maintain high clean accuracy while dominating robustness metrics.
On CIFAR-100-C, using a Wide-ResNet architecture, NoisyMix achieved 71.9% robust accuracy, a substantial lead over AugMix (66.7%) and NFM (58.7%).
Discussion and Calibration
A critical finding is that NoisyMix models are "better calibrated." Their confidence scores more accurately reflect the likelihood of being correct, which is essential for safety-critical applications. The method achieved the lowest RMS calibration error (3.9%) among all tested schemes.
While currently optimized for computer vision, the underlying principles of noise injection and consistency loss are broadly applicable. Future work aims to extend these "stability-first" concepts to other domains like scientific machine learning and climate modeling, where data is inherently noisy and reliability is paramount.
Summary updated