TL;DR
Researchers have developed a new defense against adversarial "synonym-swapping" attacks in NLP by modeling word substitutions as a convex hull in vector space. By combining this geometric approach with a sparsity-inducing regularizer, their method (ASCC) creates models that are significantly more robust and produces "plug-and-play" word embeddings that can protect standard models without additional training.
The Problem: Brittle Text Models
Deep learning models for text are notoriously brittle. An attacker can often flip a model's prediction (e.g., changing a "positive" sentiment to "negative") simply by replacing a few words with their synonyms. While these changes are nearly invisible to humans, they wreak havoc on neural networks.
Defending against these "word substitution" attacks is difficult because text is discrete, while the gradients used to train models are continuous. Previous defenses tried to bridge this gap by creating a "safety bubble" around word vectors in the form of $l_2$-balls (spheres) or hyper-rectangles (boxes). However, these shapes are often a poor fit: spheres might miss actual synonyms, and boxes often include "dead space" that represents no real word at all, leading to over-conservative models that perform poorly on clean data.
Methods: Adversarial Sparse Convex Combination (ASCC)
The authors propose Adversarial Sparse Convex Combination (ASCC). The core idea is to move away from arbitrary spheres and boxes and instead use the convex hull—the smallest convex shape that contains all possible valid word substitutions for a given word.
Key components include:
- Convex Hull Modeling: Instead of searching for any point in a radius, ASCC searches for a weighted combination of actual synonym vectors. This ensures the "perturbation space" is as tight and inclusive as possible.
- Sparsity Regularization: Because a weighted average of 50 synonyms doesn't look like a real word, the authors add an entropy-based regularization term. This pushes the optimization toward a "sparse" solution—ideally a single actual word substitution—aligning the continuous optimization process with the discrete nature of language.
- ASCC-Defense: This is an adversarial training regime where the model is trained on the "worst-case" perturbations found within these convex hulls.
Key Insights and Results
The authors evaluated their approach on two standard NLP benchmarks: IMDB (sentiment analysis) and SNLI (natural language inference). They tested these against sophisticated attack algorithms, including Genetic attacks and PWWS (Probability Weighted Word Saliency).
- Superior Robustness: ASCC-defense consistently outperformed previous state-of-the-art (SOTA) methods across different architectures (CNNs, LSTMs, and Bag-of-Words). On the IMDB dataset under Genetic attacks, ASCC achieved 79.0% accuracy, compared to the previous SOTA of 75.0% and a baseline (undefended) accuracy of just 1.0%.
- BERT Compatibility: The method also showed promise when applied to BERT, improving its robust accuracy on IMDB from 16.4% to 70.2%.
- Plug-and-Play Defense: One of the most striking findings is that the robustness "lives" within the word vectors themselves. By training word vectors using the ASCC objective, the authors created a set of "robust embeddings." When these robust word vectors were plugged into a standard, undefended LSTM, the model's robust accuracy jumped from 7.9% to 73.4% without any further adversarial training.
Limitations and Future Work
- Computational Cost: Like most adversarial training methods, ASCC requires solving an "inner maximization" problem (finding the worst-case perturbation) for every training step, which increases training time compared to standard models.
- Dependency on Substitution Sets: The defense is only as good as the predefined list of synonyms (the substitution set). If an attacker finds a valid substitution not in the set, the convex hull won't cover it.
- Future Directions: The authors suggest exploring how to automatically construct better substitution sets and investigating if these robust vectors can help with other types of distribution shifts beyond intentional adversarial attacks.
Summary updated