Abstract
Adversarial Training (AT) is a primary defense against adversarial attacks, yet standard methods often fail to account for the inherent semantic biases in untargeted attacks. We present Self-Paced Adversarial Training (SPAT), a novel framework that adaptively identifies and upweights "Hard-Class Pairs" (HCPs)—semantically similar classes that are frequently confused during attacks. By employing a hyperspherical normalized cross-entropy loss and a self-paced consistency regularization, SPAT significantly enhances model robustness across multiple benchmarks, outperforming state-of-the-art defenses against PGD and AutoAttack.
The Hard-Class Pair Problem
Deep Neural Networks (DNNs) are notoriously fragile to human-imperceptible adversarial perturbations. While Adversarial Training (AT) improves robustness by training on malicious samples, it typically treats all misclassifications with equal weight. However, research reveals that "untargeted" attacks are not truly random; they are highly biased toward semantically similar categories.
For instance, over 40% of adversarial "dog" images are misclassified as "cats" by standard models. These closely related categories form Hard-Class Pairs (HCPs). As shown in the t-SNE visualization below, these classes reside in close proximity in the feature space, making them vulnerable to "virtual targeted attacks" where an untargeted gradient descent naturally pushes samples toward the nearest incorrect class prototype.
Self-Paced Adversarial Training (SPAT)
To address this imbalance, the authors propose SPAT, which modifies the adversarial training objective through two primary components:
- Self-Paced Accuracy Loss ($L^{sp}_{acc}$): Utilizing Normalized Cross-Entropy (NCE) on a hypersphere, SPAT introduces modulating factors that adaptively upweight the loss for HCPs while downweighting "Easy-Class Pairs" (ECPs). This forces the model to learn more discriminative boundaries where they are most needed.
- Self-Paced Robustness Loss ($L^{sp}_{rob}$): A new regularization term, $L_{inc}$, penalizes inconsistencies in how clean and adversarial samples relate to class prototypes. This ensures that the semantic relationship of a sample to its neighbors remains stable under perturbation.
The "self-paced" mechanism ensures these weights are not static; they evolve with the model's learning state, focusing optimization on the most difficult boundaries at any given time.
Experimental Results
SPAT was evaluated on MNIST and CIFAR-10 benchmarks using WideResNet architectures. It consistently outperformed state-of-the-art defenses such as TRADES, MART, and Madry’s AT.
On CIFAR-10, SPAT achieved 58.33% accuracy under a 20-step PGD attack, a significant improvement over Madry’s (46.70%) and TRADES (51.82%). Against the AutoAttack (AA) ensemble, SPAT maintained a robust 48.09% accuracy.
Crucially, SPAT mitigates the "targeted" bias of untargeted attacks. As illustrated in the figure below, the concentration of misclassifications into a single "hard" class is significantly reduced compared to vanilla training, leading to a more balanced and robust feature space.
Conclusion
By recognizing that untargeted attacks are essentially a superposition of false-class prototypes weighted by semantic similarity, SPAT provides a principled way to harden models against their most likely failure modes. While future work is needed to scale this to high-resolution datasets like ImageNet, SPAT represents a significant step toward more reliable and semantically-aware adversarial defenses.
Summary updated