When evaluating whether a machine learning model is secure against adversarial attacks or distribution shifts, the standard practice has long been to report a single snapshot of performance. Typically, researchers pick an arbitrary noise budget—such as an $L_\infty$ perturbation threshold of $\epsilon = 8/255$ for image classification—and report the model's accuracy under an attack constrained to that budget.
However, this point-wise approach introduces a dangerous form of evaluation bias. Treating robustness as a binary state at a single threshold fails to capture a model's true vulnerability profile. A model that appears highly robust at a specific budget might experience a sudden, non-linear collapse in accuracy with only a marginal increase in attack strength. To truly understand a model's security, the research community has shifted toward continuous robustness evaluation, mapping a model's performance across an entire spectrum of attack intensities using security evaluation curves.
The Illusion of Single-Point Security
Why is a single accuracy metric misleading? As Göpfert et al. argue, evaluating robustness at a fixed perturbation budget provides an incomplete view of global stability. It hides critical failure thresholds where performance drops precipitously.
Consider two models that both achieve 50% adversarial accuracy at a benchmarked threshold of $\epsilon = 0.1$.
- Model A maintains roughly 50% accuracy even as $\epsilon$ increases to 0.2, degrading gracefully.
- Model B drops to 0% accuracy the moment $\epsilon$ reaches 0.11.
A point-wise evaluation treats these models as identical, masking the fact that Model B is highly brittle. Biggio et al. note that traditional static metrics fail because they do not account for varying levels of adversarial effort. True security evaluation requires characterizing how a system degrades under increasing adversarial pressure rather than relying on a single, arbitrary data point.
Mapping the Complete Vulnerability Profile
To address this, Carlini et al. established a methodological framework recommending that evaluations provide full accuracy-versus-perturbation curves. By plotting model performance as a continuous function of attack strength, researchers can observe the exact breaking point of a defense.
Accuracy
^
1.0 |-----------\
| \ <- Graceful Degradation (Robust)
| \
0.5 | \-------\
| | \ <- Catastrophic Collapse (Brittle)
| | \
0.0 +---------------+---------\---> Perturbation Budget (ε)
^
Arbitrary Threshold
This continuous mapping reveals the rate of degradation. Demontis et al. show that tracking this continuous relationship provides a holistic view of how models behave under stress. This insight extends beyond intentional adversarial attacks to natural environmental changes. For instance, Shen demonstrates that evaluating models across a continuous range of image corruption intensities exposes severe performance collapses that are completely missed by traditional benchmarks using discrete severity levels.
The Danger of Ranking Inversions
The necessity of continuous evaluation becomes even more acute when comparing different models. When researchers evaluate defenses using success rates at a single fixed perturbation level, they assume that if Model A beats Model B at threshold $\epsilon_1$, it will beat it everywhere.
Dong et al. discovered that this assumption is false: model performance rankings frequently invert as attack strength changes. A defense that appears superior at a low noise budget may become inferior when the adversary is granted a slightly higher computational or noise budget.
This threshold-specific vulnerability is not unique to adversarial noise. Li et al. observed a similar phenomenon under distribution shifts, where evaluating at a single arbitrary point of shift fails to capture how rapidly a model's accuracy plummets as the intensity of the shift scales. Without a continuous curve, researchers risk selecting models that are optimized for an arbitrary benchmark but fundamentally fragile in real-world deployments.
From Curves to Scalar Metrics: Integrating the Frontier
While curves offer rich visual intuition, comparing models or optimizing them during training requires scalar metrics. To preserve the global perspective of a security curve in a single value, researchers integrate the area under the curve or measure the geometry of the adversarial space.
Expected and Budget-Averaged Robustness
Instead of reporting accuracy at one point, Ngnawe et al. propose calculating the expected robustness, which is the area under the accuracy-perturbation curve across a continuous range of budgets. This metric captures the holistic trade-off between clean accuracy and adversarial resilience. Similarly, Melis et al. utilize budget-averaged robustness to quantify the mean detection rate across a spectrum of allowed modifications, directly measuring the system's rate of decay.
Adversarial Sparsity and Hypervolume
Other approaches look at the volume of the vulnerability landscape. Olivier et al. introduce adversarial sparsity, a metric that measures the total volume of successful perturbations. They show that even when a defense is technically "broken" (its accuracy drops to zero under strong attacks), it can still be highly valuable if it significantly increases the search effort required for an attacker to find a vulnerability.
To capture both the distance of an attack and the model's internal certainty, Guo et al. treat robustness as a multi-objective optimization problem. They calculate the Adversarial Hypervolume enclosed by the Pareto frontier of perturbation distance versus classification confidence, rewarding models that maintain high confidence and degrade gracefully under stress.
Efficient Computation via Minimum-Norm Attacks
The primary obstacle to adopting continuous evaluation is computational cost. Running a standard fixed-budget attack (like PGD) at dozens of different discrete thresholds to plot a curve is incredibly expensive.
To solve this, researchers leverage minimum-norm attacks. A standard fixed-budget attack asks: Can I find an adversarial example within a radius $\epsilon$? A minimum-norm attack flips the question: What is the smallest possible perturbation required to cause a misclassification?
As Mura et al. explain, minimum-norm optimization allows researchers to generate a complete robustness curve in a single execution. By finding the exact distance to the decision boundary for every sample in a dataset, you obtain a distribution of failure points. Sorting these distances automatically reconstructs the entire accuracy-versus-perturbation curve, eliminating the need for computationally prohibitive sweeps across fixed budgets and making continuous security profiling practical at scale.
Key Foundational Work
Adversarial Robustness Curves (2019) — contributed 4 concept elements to this direction.
secml: A Python Library for Secure and Explainable Machine Learning (2019) — contributed 3 concept elements to this direction.
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2018) — contributed 2 concept elements to this direction.