For years, the primary defense against adversarial attacks has been Adversarial Training (AT): augmenting the training set with examples perturbed by an attacker. While this significantly improves a model's resilience compared to standard training, the field has hit a stubborn performance plateau. Even on relatively simple datasets like CIFAR-10, robust accuracy rarely exceeds ~70%, leaving a "robustness ceiling" that prevents deep learning from being safely deployed in high-stakes environments like autonomous driving or medical diagnosis.
Recent research suggests this ceiling isn't just a lack of compute or data, but a fundamental limitation of how current models learn to handle conflict and distribution shifts.
The Scarcity of Training Signal
One reason adversarial training plateaus is that models eventually become "too good" at defending their own training data. Chen Chen and colleagues observe in Guided Interpolation for Adversarial Training that as training progresses, the proportion of data points that can be successfully perturbed decreases.
When a model can easily defend its training samples, the "adversarial" examples generated during training stop being challenging. Without these difficult signals, the model lacks the necessary information to further refine its decision boundaries. To counter this, the Guided Interpolation Framework (GIF) selectively creates new, "attackable" data points by interpolating between samples, effectively manufacturing the challenge needed to push past the plateau.
Theoretical Floors and Conflict Graphs
Is it possible to reach 100% robust accuracy, or is some vulnerability inherent to the data itself? Arjun Bhagoji and Prateek Mittal addressed this by establishing Lower Bounds on Cross-Entropy Loss.
They model the dataset as a "conflict graph," where an edge exists between two points if an adversary can perturb one to look like the other. If two points with different labels are too close in the input space, no classifier—no matter how advanced—can distinguish them perfectly under attack. Their work reveals a significant gap between current performance and these theoretical limits, suggesting that while some vulnerability is inherent, our current training algorithms are still underperforming.
The Out-of-Distribution Bottleneck
The robustness ceiling becomes even more apparent when models move from controlled lab settings to the real world. In OODRobustBench, Lin Li and Yifei Wang analyzed over 700 models to find a "Robustness on the Line" phenomenon.
While there is a linear correlation between in-distribution (ID) and out-of-distribution (OOD) robustness, the returns are diminishing. Improvements in ID robustness yield increasingly smaller gains in OOD scenarios. This suggests that simply scaling existing AT methods is insufficient; even a "perfectly" robust model on a training set would likely remain vulnerable to adversarial attacks in the wild due to this inherent performance ceiling.
Strategies for Bypassing the Ceiling
If we cannot achieve perfect classification, how do we build safe systems? Researchers are exploring two main paths: rejection and purification.
Rectified Rejection
Instead of forcing a model to make a guess on a highly perturbed input, Tianyu Pang and Yinpeng Dong propose that the model should simply "reject" the input. In Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart, they introduce Rectified Rejection (RR). By coupling standard confidence with a new "Rectified Confidence" (R-Con), the model can provably distinguish between inputs it is likely to classify correctly and those that are adversarial, providing a safety fallback when the robustness ceiling is reached.
Hybrid Purification
Another approach is to "clean" the input before it reaches the classifier. The CARSO framework, developed by Alessio Ansuini and Luca Bortolussi, blends adversarial training with representation-conditional purification (paper). By using the internal features of a robust classifier to guide a generative model in "purifying" the input, CARSO breaks the plateau of standard AT, achieving higher accuracy against even the most rigorous adaptive attacks.
These diverse approaches—from theoretical graphing to generative purification—all point to a shared realization: the next generation of AI security will not come from scaling standard adversarial training, but from fundamentally rethinking how models handle the limits of their own certainty.