Boosting Model Resilience via Implicit Adversarial Data Augmentation

Xiaoling Zhou, Wei Ye, Zhemg Lee, Rui Xie, Shikun Zhang, Shu-Tao Xia

Abstract

Deep Neural Networks (DNNs) are notoriously sensitive to biases in training data, such as long-tail distributions, noisy labels, and subpopulation shifts. Traditional data augmentation techniques often struggle to bridge the gap between biased training sets and real-world testing scenarios. We present Implicit Adversarial Data Augmentation (IADA), a novel framework that enhances model resilience by perturbing deep features with sample-wise adversarial and anti-adversarial strategies. By deriving a mathematically grounded surrogate loss for infinite augmentations, IADA provides a highly efficient, "GPU-friendly" alternative to explicit sample generation. Furthermore, a meta-learning framework adaptively tunes the augmentation strength for each sample, allowing the model to handle "hard" and "noisy" samples differently. IADA achieves state-of-the-art results across diverse benchmarks, including CIFAR-10-LT, ImageNet-LT, and Waterbirds.

Introduction

Modern machine learning models frequently encounter datasets with significant biases. Long-tail distributions, where a few classes dominate the training set, often lead to poor performance on minority classes. Noisy labels and subpopulation shifts—where models rely on spurious correlations—further degrade reliability. While data augmentation is a standard remedy, existing methods often apply uniform strategies across all samples, failing to account for the unique difficulty of individual data points.

IADA addresses these challenges by moving augmentation into the feature space and employing an "implicit" approach. Instead of generating new images, it applies perturbations directly to internal representations. This allows for more targeted interventions, such as making learning harder for well-represented classes or easier for samples with potentially incorrect labels.

Conceptual illustration of (a) feature-space perturbations relative to the classification boundary and (b) the impact of augmentation strength on majority and minority class distributions.

Methodology

The IADA framework is built upon three core pillars: feature-space perturbations, an implicit surrogate loss, and a meta-learning adaptation mechanism.

Feature-Space Perturbations

IADA utilizes two types of perturbations in the deep feature space:

  • Adversarial Perturbations: These push features toward the decision boundary, increasing the difficulty of the task and forcing the model to learn more robust boundaries. This is particularly effective for minority classes in long-tail scenarios.
  • Anti-adversarial Perturbations: These push features away from the boundary, making learning "easier." This is crucial for mitigating the impact of noisy labels, effectively allowing the model to ignore or down-weight unreliable samples.

Implicit Surrogate Loss and Regularization

A key innovation of IADA is the derivation of an implicit loss function. The authors prove that augmenting a feature an infinite number of times with these perturbations is mathematically equivalent to optimizing a specific robust loss. This "implicit" approach eliminates the computational and memory overhead of generating and storing billions of augmented samples, making it highly efficient for large-scale training. The derived loss naturally decomposes into three regularization terms that promote generalization (class compactness), robustness (sharpened boundaries), and fairness (balanced influence between majority and minority classes).

Meta-IADA Framework

To achieve sample-wise adaptability, IADA employs a meta-learning framework. A small, clean "metadata" set is used to guide a perturbation network. This network adaptively determines the optimal augmentation strength and direction for every training sample based on its current loss and margin. For instance, it learns to apply stronger adversarial pressure to tail-class samples while using anti-adversarial strategies for potentially noisy data.

The Meta-IADA framework architecture, illustrating the flow from feature perturbations to the implicit loss and the meta-learning-based strategy calibration.

Experimental Results

IADA was evaluated across four major biased learning categories using a wide array of benchmarks:

  • Long-Tail Learning: On CIFAR-10-LT (100:1 imbalance), IADA achieved 84.01% accuracy, outperforming the previous state-of-the-art (BKD) by 1.51%.
  • Noisy Label Learning: In scenarios with 40% label noise on CIFAR-10, IADA maintained a high accuracy of 91.99%, compared to just 70.78% for standard training.
  • Subpopulation Shift: On the "Waterbirds" dataset, which tests for spurious correlations, IADA achieved a 92.5% worst-group accuracy, significantly surpassing GroupDRO (90.6%).
  • Generalized Long-Tail: The method also showed superior performance on complex datasets like ImageNet-GLT and MSCOCO-GLT, demonstrating its versatility across both image and text data.

Conclusion

Implicit Adversarial Data Augmentation (IADA) provides a theoretically grounded and computationally efficient solution for building resilient models. By leveraging sample-wise adaptability and implicit feature-space perturbations, IADA effectively addresses the most common pitfalls of biased training data. Its ability to handle long-tails, noise, and shifts within a single framework makes it a powerful tool for modern deep learning applications.

Summary updated

BibTeX
@inproceedings{zhou2024boosting,
  author = {Xiaoling Zhou and Wei Ye and Zhemg Lee and Rui Xie and Shikun Zhang and Shu-Tao Xia},
  title = {{Boosting Model Resilience via Implicit Adversarial Data Augmentation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=z0J3fh8dkW},
}
Figure