To find an adversarial example, we usually follow the gradient of a loss function. If we want a model to misclassify a "panda" as a "gibbon," we tweak the image pixels in the direction that increases the "gibbon" probability.
However, researchers have discovered a fundamental flaw in this process: the very loss functions we use to train models—like Cross-Entropy—are often the worst tools for attacking them. As a model becomes more confident in a prediction, the gradient of the Cross-Entropy loss vanishes. This "gradient saturation" creates a flat landscape where the optimizer can no longer find a path to improve the attack, leading to a false sense of security and poor transferability across models.
The Probability Trap
The core of the problem lies in the mathematical shape of standard loss functions. In On Success and Simplicity: A Second Look at Transferable Targeted Attacks, Zhengyu Zhao and colleagues point out that as the probability of a target class approaches 1, the gradient of the Cross-Entropy loss approaches 0.
For a classifier, this is a feature: once it is certain of a label, it stops changing its weights. But for an attacker, this is a "probability trap." If an attack successfully nudges an image just over the decision boundary of a surrogate model, the gradient disappears, and the optimization stops. This results in adversarial examples that are "barely" adversarial. While they might fool the surrogate model, they lack the "confidence" or "margin" needed to fool a different, black-box model. Juanjuan Weng and others argue in Logit Margin Matters that this logit saturation prevents the discovery of robust adversarial features that could transfer to different architectures.
Gradient Imbalance and Masking
Gradient saturation doesn't just stop attacks; it can actively mislead them. Xingjun Ma identifies a phenomenon called Imbalanced Gradients (IG) in Imbalanced Gradients: A New Cause of Overestimated Adversarial Robustness. When a loss function has multiple components (e.g., a classification term and a regularization term), one term can dominate the gradient signal, drowning out the others.
This creates a form of "gradient masking." The model appears robust because the standard attack's gradient is pointing in a suboptimal direction, even though a successful adversarial perturbation exists nearby. Similarly, in pixel-wise tasks like semantic segmentation, Kira Maag found that optimizers often waste "gradient energy" on pixels that are already successfully misclassified, leaving resistant pixels untouched.
Strategies for Unstoppable Optimization
To bypass these flat regions of the loss surface, researchers have proposed several "gradient-boosting" strategies:
- Logit and Margin Losses: Instead of optimizing for probability, Zhengyu Zhao suggests using the raw Logit loss. Unlike Cross-Entropy, the gradient of the logit value does not vanish as confidence increases, allowing the attack to keep refining the perturbation. Ruize Gao takes this further with the Minimum-Margin (MM) attack, which focuses on the distance to the decision boundary rather than class probability, achieving the reliability of complex attacks at a fraction of the cost.
- Logit Calibration: In Logit Margin Matters, Juanjuan Weng introduces temperature scaling and adaptive margins. By "softening" the logits, these techniques prevent the loss from saturating too early, forcing the attack to find deeper, more transferable vulnerabilities.
- Latent Feature Targeting: If the final output layer is saturated, why not attack the middle? Yunrui Yu developed LAFEAT, which targets internal (latent) features. This "pierces" through defenses that rely on output-layer gradient masking. Similarly, Yixiang Qiu uses intermediate features of GANs to improve model inversion attacks, bypassing the gradient stall that occurs when a target model is highly confident.
- Dynamic Weighting: To solve the imbalance in dense tasks, Margret Keuper and Steffen Jung proposed CosPGD, which uses a cosine-alignment score to dynamically scale losses across an image. This ensures the optimizer focuses its budget on the most resistant regions.
Why It Works: Pushing Beyond the Boundary
The intuition behind these methods is that a "good" adversarial example shouldn't just be a mistake; it should be a confident mistake.
By preventing vanishing gradients, these techniques allow the optimization to continue long after the model has been "fooled." This pushes the adversarial example deeper into the target class's territory. As Mauro Barni and colleagues show in Increased-confidence adversarial examples, increasing this "confidence margin" is the key to making attacks transfer between different models. When we fix the gradient, we reveal the true, underlying fragility of the neural network's decision boundaries.