ARCH: Efficient Adversarial Regularized Training with Caching

Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

Abstract

Adversarial Regularization (AR) is a potent technique for enhancing model generalization in Natural Language Processing (NLP). However, traditional AR methods like Projected Gradient Descent (PGD) are computationally intensive, often slowing training by 3-5x due to the need for repeated perturbation calculations. This paper introduces ARCH (Adversarial Regularization with CacHing), an efficient framework that reuses perturbations across multiple training epochs. By combining a caching mechanism with K-Nearest Neighbors (KNN) for memory optimization and Exponential Moving Average (EMA) for smooth updates, ARCH achieves a 50-70% reduction in computational overhead. Empirically, ARCH not only accelerates training but also improves model stability and performance across Neural Machine Translation (NMT) and Natural Language Understanding (NLU) benchmarks.

1. Introduction

Adversarial Regularization improves robustness by training models on "worst-case" perturbations of input embeddings. Despite its benefits, the "calculate-every-time" nature of current methods leads to significant computational costs and high gradient variance. We propose ARCH to address these inefficiencies by decoupling perturbation generation from every training step.

2. Methodology

ARCH introduces three core innovations to streamline adversarial training:

  • Perturbation Caching: Instead of per-epoch calculation, perturbations are computed once every several epochs (e.g., every 15 or 20) and stored in a cache.
  • EMA Updates: To maintain relevance as the model evolves, new perturbations are blended with cached ones using an Exponential Moving Average (EMA), ensuring smooth transitions and helping the model adapt.
  • KNN-based Memory Efficiency: To handle large datasets like WMT without massive memory overhead, ARCH caches only a fraction (e.g., 10%) of perturbations. For the remainder, it approximates noise by averaging the cached perturbations of the K-nearest neighbors in the embedding space.

3. Experimental Evaluation

We evaluated ARCH on NMT (IWSLT, WMT) and NLU (GLUE) tasks, comparing it against state-of-the-art methods like SMART, FreeLB, and R3F.

3.1 Efficiency and Speedup

ARCH demonstrates substantial time savings across all benchmarks. On the high-resource WMT’16 En-De dataset, ARCH reduced training time from 576 GPU hours (using the SMART method) to just 176 GPU hours.

Training time comparison (in seconds) showing ARCH's significant efficiency gains over baseline methods like FreeLB and SMART.

3.2 Performance and Stability

Contrary to the intuition that "fresh" noise is necessary, ARCH's "stale" perturbations act as a more stable regularizer. This stability is reflected in the gradient norms during training, where ARCH significantly dampens the volatility seen in traditional methods.

Gradient norm visualization demonstrating ARCH's superior training stability (lower variance) compared to the volatile SMART and R3F methods.

In terms of performance, ARCH improved BLEU scores by an average of 0.5 points in NMT and achieved a GLUE average of 84.5, outperforming BERT-base (81.5) and other AR methods like SMART (83.8) and R3F (83.7).

4. Conclusion

ARCH proves that stability in adversarial training can be more beneficial than the absolute freshness of perturbations. By caching and reusing noise, ARCH provides a scalable and efficient path for applying adversarial regularization to large-scale NLP models. While current results focus on NLP, the success of this caching strategy suggests potential for future applications in other high-dimensional domains.

Summary updated

BibTeX
@inproceedings{zuo2021arch,
  author = {Simiao Zuo and Chen Liang and Haoming Jiang and Pengcheng He and Xiaodong Liu and Jianfeng Gao and Weizhu Chen and Tuo Zhao},
  title = {{ARCH: Efficient Adversarial Regularized Training with Caching}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=SOiPgCQcDWO},
}
Figure