Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies

Brian R. Bartoldson, James Diffenderfer, Konstantinos Parasyris, Bhavya Kailkhura

TL;DR

Researchers have developed the first scaling laws for adversarial training, revealing that current state-of-the-art models are compute-inefficient and over-parameterized. By optimizing resource allocation, they achieved a new SOTA on CIFAR-10, but discovered a fundamental "robustness ceiling" at ~90% caused by adversarial attacks that semantically change images, fooling humans and machines alike.

What Problem Does This Paper Address?

For over a decade, the machine learning community has struggled with adversarial examples: tiny, imperceptible perturbations that cause high-performing models to fail. On the benchmark CIFAR-10 dataset, "clean" accuracy is essentially solved at ~100%, yet robust accuracy against $\ell_\infty$ attacks has stubbornly plateaued around 70%.

The authors ask a fundamental question: Why is this problem still unsolved? Is it a lack of compute and data (a scaling problem), or is there a flaw in how we define and measure robustness?

Methods

The authors bridge the gap between adversarial training (AT) and the "scaling law" literature popularized by Large Language Models. Their approach involves:

  1. Adversarial Scaling Laws: They developed three parametric models to predict robust loss based on model size ($N$), dataset size ($D$), and—crucially—synthetic data quality (measured by FID).
  2. Massive-Scale Training: To fit these laws, they conducted over 230 training runs, consuming $10^{22}$ FLOPs. They utilized synthetic datasets of up to 300 million unique images generated by diffusion models (EDM, PFGM++).
  3. Compute-Optimal Analysis: Similar to the "Chinchilla" study for LLMs, they identified the optimal ratio of model size to data size for a given compute budget.
  4. Human-Alignment Study: They conducted a controlled study where humans and GPT-4 classified the specific adversarial images that fooled their best-performing model to determine if the "ground truth" labels were still valid.

Datasets

  • CIFAR-10: The primary benchmark for training and evaluation.
  • Synthetic CIFAR-10: Generated using EDM, PFGM++, and Discriminator Guidance (DG) to vary data quality (FID ranges from 1.65 to 35.54).
  • Human Evaluation Set: A subset of 2,629 adversarial images generated by AutoAttack that successfully tricked the authors' SOTA model.

Key Insights and Ideas

  • The Efficiency Gap: Current SOTA methods are significantly compute-inefficient. They tend to use models that are too large for the amount of data they are trained on. The authors show that training smaller models on more data yields better robustness for the same FLOP budget.
  • The 90% Asymptote: Their scaling laws predict that even with infinite compute and perfect synthetic data, robust accuracy on CIFAR-10 will plateau near 90%. Scaling alone cannot reach 100%.
  • Invalid Adversarial Data: The "ceiling" exists because $\ell_\infty$-constrained attacks often produce invalid data. At high levels of robustness, the attack perturbations (though mathematically small) are sufficient to change the semantic content of the image. For example, a "horse" might be perturbed until it actually looks like a "dog" to a human observer.
Scaling laws predict a performance plateau and show the "horse-to-dog" semantic shift.
  • Human-Model Alignment: Surprisingly, human performance on these adversarial images also plateaus near 90%. This suggests that models are becoming so robust that the only way for an attack to succeed is to break the underlying logic of the label itself.

Key Results

  • New SOTA: The authors achieved a new state-of-the-art robust accuracy of 73.71% on CIFAR-10 (AutoAttack, $\epsilon=8/255$) using a WideResNet-94-16 trained on 500M samples.
  • Efficiency Gains: They matched the previous SOTA robustness while using 20% fewer training FLOPs and 70% fewer inference FLOPs by following their scaling law recommendations.
  • Human Performance: The study estimated average human accuracy on AutoAttack data at 90.46%. When "invalid" images (those humans couldn't agree on) were removed, the model's robust accuracy jumped from 73.71% to 79.49%.
The "compute-efficient region" identified by the authors' scaling laws.

Limitations and Future Work

  • Scope: The study focuses primarily on CIFAR-10 and WideResNet architectures. While foundational, these findings need validation on higher-resolution datasets like ImageNet and different architectures like Vision Transformers (ViTs).
  • Human Study Size: The human evaluation was small-scale (3 participants + GPT-4). The authors have released an online quiz to gather more community data.
  • Rethinking Attacks: The most significant takeaway is a call to action: the field must move beyond simple $\ell_p$ norm constraints. Future research should focus on developing validity-aware attacks that ensure the adversarial image remains a true member of its original class.

Summary updated

Abstract

This paper revisits the simple, long-studied, yet still unsolved problem of making image classifiers robust to imperceptible perturbations. Taking CIFAR10 as an example, SOTA clean accuracy is about $100$%, but SOTA robustness to $\ell_{\infty}$-norm bounded perturbations barely exceeds $70$%. To understand this gap, we analyze how model size, dataset size, and synthetic data quality affect robustness by developing the first scaling laws for adversarial training. Our scaling laws reveal inefficiencies in prior art and provide actionable feedback to advance the field. For instance, we discovered that SOTA methods diverge notably from compute-optimal setups, using excess compute for their level of robustness. Leveraging a compute-efficient setup, we surpass the prior SOTA with $20$% ($70$%) fewer training (inference) FLOPs. We trained various compute-efficient models, with our best achieving $74$% AutoAttack accuracy ($+3$% gain). However, our scaling laws also predict robustness slowly grows then plateaus at $90$%: dwarfing our new SOTA by scaling is impractical, and perfect robustness is impossible. To better understand this predicted limit, we carry out a small-scale human evaluation on the AutoAttack data that fools our top-performing model. Concerningly, we estimate that human performance also plateaus near $90$%, which we show to be attributable to $\ell_{\infty}$-constrained attacks' generation of invalid images not consistent with their original labels. Having characterized limiting roadblocks, we outline promising paths for future research.
BibTeX
@inproceedings{bartoldson2024adversarial,
  author = {Brian R. Bartoldson and James Diffenderfer and Konstantinos Parasyris and Bhavya Kailkhura},
  title = {{Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=HQtTg1try7},
}
Figure