Token-Aware Virtual Adversarial Training in Natural Language Understanding

Linyang Li, Xipeng Qiu

Introduction

Neural networks in Natural Language Processing (NLP) are notoriously brittle; small, intentional perturbations to input data—known as adversarial attacks—can easily degrade model performance. Adversarial Training (AT) has emerged as a primary defense, involving the addition of noise during training to improve robustness. In NLP, where inputs are discrete, researchers often employ Virtual Adversarial Training (VAT) to apply noise to continuous word embeddings. However, standard VAT methods suffer from two primary limitations: they reset noise randomly for every batch (the initialization problem) and apply a uniform noise budget across all tokens regardless of their semantic importance (the constraint problem).

Methodology: The TA-VAT Approach

This paper introduces Token-Aware Virtual Adversarial Training (TA-VAT), a method designed to craft more precise and stable adversarial noise by accounting for token-specific characteristics. TA-VAT introduces two core innovations:

  1. Global Perturbation Vocabulary ($V$): Instead of initializing noise randomly for each training step, TA-VAT maintains a global dictionary that stores and accumulates perturbations for every word in the vocabulary. This provides a "warm start" for noise generation, reducing random jitter and stabilizing the training process.
  2. Token-Level Constraints and Scaling: Rather than applying a single noise budget to an entire sequence, TA-VAT applies constraints at the individual token level. It utilizes a Scaling Index ($n^i$) based on the gradient magnitude of each token. This allows "important" tokens—those that significantly influence the model's output—to receive a larger perturbation budget, while trivial tokens are more tightly constrained.

Experimental Results

TA-VAT was evaluated across the GLUE benchmark, showing consistent improvements over baseline models like BERT and ALBERT. By focusing perturbations on the most critical parts of the sentence, the model learns a more robust representation of language.

Table 1: Performance comparison of TA-VAT against BERT-BASE and ALBERT-xxlarge baselines across GLUE tasks.

As shown in the results above, TA-VAT significantly outperforms standard BERT and state-of-the-art adversarial methods like FreeLB. On the GLUE benchmark, TA-VAT boosted the BERT-base score from 78.3 to 80.9 on the test set and improved the ALBERT-xxlarge model to 90.9 on the development set.

Data Efficiency and Robustness

One of the most striking findings is TA-VAT's performance in low-resource settings. The method is particularly effective when training data is scarce, as the global perturbation vocabulary allows the model to leverage "memory" of token sensitivity across different contexts.

Figure 1: Comparison of model performance across varying training set sizes, demonstrating TA-VAT's superior data efficiency.

Beyond standard benchmarks, TA-VAT demonstrates enhanced robustness against specialized attacks. When subjected to the TextFooler attack, TA-VAT increased the "Attacked Accuracy" and nearly doubled the number of queries required for an attacker to successfully break the model. Furthermore, the learned perturbation vocabulary proved to be transferable; applying the learned noise to a vanilla model's embeddings improved its performance even without additional training.

Conclusion

TA-VAT addresses the inherent limitations of uniform adversarial training in NLP by introducing token-awareness. By maintaining a global memory of perturbations and scaling noise based on token importance, TA-VAT achieves superior performance on GLUE, better data efficiency, and increased robustness against adversarial attacks. While the method requires additional computational steps and memory to maintain the perturbation vocabulary, the gains in stability and accuracy make it a compelling approach for robust natural language understanding.

Summary updated

Abstract

Gradient-based adversarial training is widely used in improving the robustness of neural networks, while it cannot be easily adapted to natural language processing tasks since the embedding space is discrete. In natural language processing fields, virtual adversarial training is introduced since texts are discrete and cannot be perturbed by gradients directly. Alternatively, virtual adversarial training, which generates perturbations on the embedding space, is introduced in NLP tasks. Despite its success, existing virtual adversarial training methods generate perturbations roughly constrained by Frobenius normalization balls. To craft fine-grained perturbations, we propose a Token-Aware Virtual Adversarial Training method. We introduce a token-level accumulated perturbation vocabulary to initialize the perturbations better and use a token-level normalization ball to constrain these perturbations pertinently. Experiments show that our method improves the performance of pre-trained models such as BERT and ALBERT in various tasks by a considerable margin. The proposed method improves the score of the GLUE benchmark from 78.3 to 80.9 using BERT model and it also enhances the performance of sequence labeling and text classification tasks.
BibTeX
@inproceedings{li2021token,
  author = {Linyang Li and Xipeng Qiu},
  title = {{Token-Aware Virtual Adversarial Training in Natural Language Understanding}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i9.17022},
}
Figure