Introduction
Deep Neural Networks (DNNs) are highly susceptible to adversarial attacks, where subtle, intentional perturbations to input data lead to incorrect model predictions. While well-documented in computer vision, crafting adversarial examples in Natural Language Processing (NLP) is significantly more challenging due to the discrete nature of text. Unlike images, where pixel values can be adjusted continuously, text requires discrete word swaps. This introduces two primary difficulties: an exponential search space and the need to maintain semantic validity. Swapping "she" for "he" might flip a sentiment classifier's output, but it also changes the ground-truth label, rendering the attack invalid. Furthermore, random synonym replacement often results in ungrammatical or nonsensical sentences. This paper explores a novel framework that treats word-level adversarial attacking as a combinatorial optimization problem to overcome these hurdles.
Sememe-based Word Substitution
To address the search space quality, the authors leverage sememes—the smallest semantic units of language. By using the HowNet knowledge base, they identify candidate substitutes that share the same sememes as the original word.
This approach offers a "Goldilocks" solution: it provides more candidates than traditional synonym sets (like WordNet) while maintaining higher semantic precision than word embeddings (like GloVe), which often group words with opposite meanings (e.g., "good" and "bad") due to similar contextual usage. On average, sememes provided 10-13 substitutes per word compared to only 3 for synonyms, significantly expanding the search space without sacrificing naturalness.
Combinatorial Optimization via PSO
Finding the optimal combination of word substitutes to fool a model is a complex optimization task. The authors employ Particle Swarm Optimization (PSO), a metaheuristic inspired by the collective behavior of bird flocking. In this context, a "swarm" of potential adversarial sentences moves through the discrete search space. Each "particle" tracks its own best result and the global best found by the swarm, moving toward successful configurations by probabilistically swapping words. This discrete adaptation of PSO includes mutation mechanisms to avoid local optima, proving more efficient than traditional Genetic Algorithms (GA).
Experimental Evaluation
The framework was tested against BiLSTM and BERT models on three benchmark datasets: IMDB (sentiment analysis), SST-2 (short-sentence sentiment), and SNLI (natural language inference).
The results demonstrate the superiority of the Sememe+PSO approach:
- High Success Rates: The attack achieved near-perfect success rates, reaching 100% on IMDB against BiLSTM and 98.7% against BERT.
- Efficiency: As shown in Figure 2, Sememe+PSO consistently outperforms synonym-based methods and Genetic Algorithms, achieving higher success rates with smaller population sizes and fewer iterations. This efficiency is critical when attacking large models like BERT, where each evaluation is computationally expensive.
- Text Quality: Human evaluations confirmed that the generated adversarial examples maintained high naturalness and label consistency. On the IMDB dataset, the model only required changing approximately 3.7% of words to successfully flip the prediction.
Conclusion
By framing adversarial attacks as combinatorial optimization, this research provides a robust method for evaluating the vulnerabilities of state-of-the-art NLP models. The combination of sememe-based substitution and PSO ensures that attacks are not only effective but also preserve the linguistic integrity of the original text. These findings pave the way for developing more resilient models through adversarial training and the potential integration of sememe knowledge into model architectures.
Summary updated