Abstract
For many real-world applications, obtaining stable and robust statistical performance is more important than simply achieving state-of-the-art predictive test accuracy, and thus robustness of neural networks is an increasingly important topic. Relatedly, data augmentation schemes have been shown to improve robustness with respect to input perturbations and domain shifts. Motivated by this, we introduce NoisyMix, a novel training scheme that promotes stability as well as leverages noisy augmentations in input and feature space to improve both model robustness and in-domain accuracy. NoisyMix produces models that are consistently more robust and that provide well-calibrated estimates of class membership probabilities. We demonstrate the benefits of NoisyMix on a range of benchmark datasets, including ImageNet-C, ImageNet-R, and ImageNet-P. Moreover, we provide theory to understand implicit regularization and robustness of NoisyMix.
Introduction
While deep learning models excel at recognizing clear, high-quality images, they are notoriously "brittle." A model that identifies a stop sign perfectly in sunny weather might fail completely if it’s snowing or if the camera lens is slightly blurry. These "common corruptions" (e.g., noise, blur, weather) and "domain shifts" (e.g., artistic renditions like sketches or cartoons) represent a major hurdle for deploying AI in safety-critical areas like autonomous driving or medical imaging.
Current methods often face a trade-off: adversarial training, designed to thwart malicious attacks, frequently decreases a model's ability to handle natural corruptions. Conversely, standard data augmentation often lacks the "input stability" required for reliable real-world performance. NoisyMix addresses this gap by providing a training framework that enhances resilience without sacrificing standard accuracy.
Methodology
NoisyMix is a training scheme that acts as a "triple threat" against model brittleness by integrating three main components:
- Diverse Augmentations: It utilizes the
AugmentAndMixapproach, which stochastically layers various transformations (rotating, shearing, color-shifting) to create highly varied versions of the training data. - Noisy Feature Mixup (NFM): Building on the popular "Mixup" technique (which blends pairs of images and labels), NFM adds random noise to the model's internal feature representations. This prevents the model from over-relying on specific, fragile patterns.
- Stability Training via JSD: The framework employs a consistency loss using Jensen-Shannon Divergence (JSD). This mathematically forces the model to produce similar probability estimates whether it is looking at a clean image or a noisy, augmented version of that same image.
Theoretically, minimizing the NoisyMix loss is equivalent to minimizing an upper bound on a combination of adversarial loss and stability objectives, providing a principled foundation for its regularization effects.
Experimental Setup
The researchers evaluated NoisyMix on several rigorous benchmarks designed to test the limits of model robustness:
- ImageNet-C: 15 types of synthetic corruptions at five levels of severity.
- ImageNet-R: Naturally occurring domain shifts, such as artistic renditions (graffiti, origami, cartoons).
- ImageNet-P: Perturbation sequences used to measure "flip probability"—how often a model changes its prediction when an image is slightly shifted or rotated.
- CIFAR-10-C & CIFAR-100-C: Corrupted versions of standard small-image datasets.
Key Results and Insights
NoisyMix consistently outperformed standard baselines and previous state-of-the-art methods like AugMix across all major benchmarks.
Performance Highlights
- ImageNet-C: Achieved 52.3% robust accuracy, a significant jump over the 39.2% baseline and 48.3% for AugMix.
- ImageNet-P: Reduced the Mean Flip Rate (mFP)—a measure of instability—from 58.0% in the baseline down to 28.5%.
- CIFAR-100-C: On a Wide-ResNet architecture, NoisyMix reached 71.9% robust accuracy, outperforming the next best method by over 5%.
Key Insights
- The "Low-Frequency" Advantage: While adversarial attacks often hide in high-frequency noise, common corruptions like fog or blur are low-frequency changes. NoisyMix is specifically tuned to handle these shifts where adversarial training often fails.
- Robustness Transfers: Robustness is "contagious." A model trained with NoisyMix on ImageNet can be fine-tuned for a different task (like CIFAR) and still retain its superior resilience to noise.
- Better Calibration: NoisyMix significantly improves model calibration, achieving the lowest RMS calibration error (3.9%) among tested schemes. This means confidence scores more accurately reflect the true likelihood of correctness.
- Component Synergy: Ablation studies show that removing any one component (noise, mixup, or JSD loss) leads to a drop in performance, indicating that these elements work together to "wash out" non-generalizable features.
Conclusion
NoisyMix demonstrates that combining diverse augmentations with internal noise injection and stability training creates models that are significantly more resilient to real-world distortions. While currently focused on computer vision, the underlying principles of stability and noise injection offer a promising path for other domains. Future work will explore adapting these concepts for language models and addressing the remaining trade-offs between common corruption robustness and adversarial attack resilience.
Summary updated