Abstract
Adversarial Regularization (AR) has emerged as a powerful technique for improving the generalization of deep learning models. However, standard AR formulations typically rely on a zero-sum min-max game, which treats the model and the attacker as equal participants. This symmetry often results in "over-strong" attacks that degrade the model's performance on clean data. We propose Stackelberg Adversarial Regularization (SALT), which reframes the AR process as a Stackelberg game. In this framework, the model acts as a leader that anticipates the attacker's moves. By employing unrolled optimization and Hessian-vector products, SALT achieves superior generalization and model calibration. Experimental results across Natural Language Understanding (NLU) and Neural Machine Translation (NMT) tasks demonstrate that SALT consistently outperforms state-of-the-art AR baselines.
Introduction
The goal of Adversarial Regularization is to find model parameters that are robust to small input perturbations. Traditionally, this is modeled as a min-max problem where an attacker maximizes loss and a defender minimizes it. While effective, this zero-sum setup assumes the attacker and defender have equal priority. In practice, this can lead to over-regularization, where the model focuses so much on defending against aggressive perturbations that it fails to fit the original, clean data distribution. This phenomenon is particularly acute in Natural Language Processing (NLP), where the discrete nature of text makes defining appropriate perturbation sets challenging.
The SALT Framework: A Strategic Approach
SALT shifts the paradigm from a simultaneous zero-sum game to a hierarchical Stackelberg game. In this formulation, the model (the leader) optimizes its parameters by anticipating the optimal response of the perturbation generator (the follower).
Unrolled Optimization
To implement this anticipation, SALT utilizes unrolled optimization. The leader's gradient calculation involves differentiating through the follower's optimization steps. This "look-ahead" mechanism allows the model to find a more stable optimization path. To manage the computational complexity of second-order derivatives, the authors employ Hessian-vector products (Hvp) via finite differences, ensuring that the training overhead remains manageable—comparable to approximately two standard backpropagation passes.
Experimental Results
The researchers evaluated SALT on a diverse set of NLP benchmarks, including the GLUE suite for NLU and several IWSLT/WMT datasets for NMT.
Performance on NLU and NMT
A BERT-base model trained with SALT achieved an average score of 84.5 on the GLUE development set, surpassing the performance of a vanilla BERT-large model (84.0) despite having significantly fewer parameters. In translation tasks, SALT provided improvements of up to 2.0 BLEU points over standard Transformer baselines in low-resource settings.
Reducing Optimization Bias
One of the primary advantages of SALT is its ability to fit clean data more effectively than standard AR methods like SMART. By anticipating the attack rather than just reacting to it, the model avoids the underfitting typically caused by over-strong adversarial perturbations.
Robustness and Calibration
SALT exhibits remarkable stability with respect to hyperparameters. As shown in the sensitivity analysis, SALT maintains high performance even as the perturbation strength increases, whereas competing methods often see performance degradation. Furthermore, SALT-trained models are better calibrated, meaning their predicted confidence levels more accurately reflect their actual accuracy, reducing the common issue of overconfidence in deep neural networks.
Conclusion
By reframing adversarial training as a Stackelberg game, SALT provides a more principled and effective way to regularize NLP models. The strategic advantage of anticipating the attacker allows for better fit on clean data, increased robustness to hyperparameters, and improved model calibration. While it introduces some computational overhead, the significant performance gains—such as BERT-base outperforming BERT-large—make it a compelling approach for large-scale NLP training.
Summary updated