ARCH: Efficient Adversarial Regularized Training with Caching

Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

Introduction

Adversarial Regularization (AR) has emerged as a critical technique for enhancing the generalization and robustness of Natural Language Processing (NLP) models. By introducing small, "worst-case" perturbations to input embeddings, AR encourages models to maintain stable outputs despite minor input variations. However, standard AR implementations face two significant hurdles: extreme computational overhead and optimization instability.

Conventional methods like Projected Gradient Descent (PGD) require multiple forward and backward passes per sample in every epoch, often slowing training by 3 to 10 times. Furthermore, recalculating perturbations every epoch against a shifting model leads to high gradient variance, which can hinder convergence. This paper introduces ARCH (Adversarial Regularization with CacHing), a framework designed to overcome these bottlenecks through efficient engineering and algorithmic stability.

The ARCH Framework

ARCH tackles the limitations of traditional AR through three core innovations:

  • Perturbation Caching: Instead of re-calculating perturbations at every step, ARCH generates them once and stores them in a cache. These perturbations are reused across multiple training epochs and only updated periodically (e.g., every 15-20 epochs). This "stability over freshness" approach allows the model to adapt more deeply to specific noise patterns.
  • Exponential Moving Average (EMA) Updates: When perturbations are eventually updated, ARCH employs an EMA strategy to blend new calculations with existing cached values. This smoothing mechanism prevents abrupt changes in the noise distribution, further stabilizing the training process.
  • KNN-based Memory Optimization: To mitigate the massive RAM requirements of caching perturbations for large-scale datasets (which can exceed 100GB), ARCH utilizes a K-Nearest Neighbors (KNN) strategy. By caching only a small fraction (e.g., 1-10%) of perturbations and reconstructing others on-the-fly via neighbor averaging in the embedding space, ARCH maintains performance while drastically reducing the memory footprint.

Experimental Results

Computational Efficiency

ARCH achieves dramatic reductions in training time compared to state-of-the-art AR methods like SMART and FreeLB. On large-scale benchmarks such as WMT’16, ARCH reduced GPU hours from 576 (SMART) to just 176, representing a nearly 70% speedup.

Comparison of training time (in seconds) across different adversarial training methods. ARCH demonstrates a significant reduction in computational cost compared to FreeLB and SMART.

Model Performance and Stability

Despite the use of "stale" cached perturbations, ARCH consistently outperforms baselines. In Neural Machine Translation (NMT) tasks, it provided an average improvement of roughly 0.5 BLEU points. On the GLUE benchmark, ARCH raised the average score by 0.7 over existing methods.

A key insight of this work is that caching reduces stochastic gradient variance. By exposing the model to consistent perturbations over several epochs, the optimization landscape becomes smoother and less volatile than methods that introduce fresh noise every epoch.

Visualization of gradient norms during training. ARCH exhibits significantly lower and more stable gradient variance compared to SMART and R3F, facilitating better convergence.

Conclusion

ARCH demonstrates that adversarial training does not have to be prohibitively expensive or unstable. By leveraging perturbation caching and KNN-based reconstruction, ARCH provides a scalable path for applying robust regularization to large-scale NLP models. The findings highlight that data-dependent, stable noise is often more informative for model generalization than high-variance, frequently updated perturbations.

Summary updated

Abstract

Adversarial regularization can improve model generalization in many natural language processing tasks. However, conventional approaches are computationally expensive since they need to generate a perturbation for each sample in each epoch. We propose a new adversarial regularization method ARCH (adversarial regularization with caching), where perturbations are generated and cached once every several epochs. As caching all the perturbations imposes memory usage concerns, we adopt a K-nearest neighbors-based strategy to tackle this issue. The strategy only requires caching a small amount of perturbations, without introducing additional training time. We evaluate our proposed method on a set of neural machine translation and natural language understanding tasks. We observe that ARCH significantly eases the computational burden (saves up to 70% of computational time in comparison with conventional approaches). More surprisingly, by reducing the variance of stochastic gradients, ARCH produces a notably better (in most of the tasks) or comparable model generalization. Our code is available at https://github.com/SimiaoZuo/Caching-Adv.
BibTeX
@inproceedings{zuo2021arch,
  author = {Simiao Zuo and Chen Liang and Haoming Jiang and Pengcheng He and Xiaodong Liu and Jianfeng Gao and Weizhu Chen and Tuo Zhao},
  title = {{ARCH: Efficient Adversarial Regularized Training with Caching}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=SOiPgCQcDWO},
}
Figure