TL;DR
This paper introduces the "Hybrid KeyToken Attack," a novel method for generating stealthy adversarial attacks on NLP models. By combining explainability techniques (SHAP) with iterative search strategies, it achieves high success rates in flipping classification labels with significantly fewer queries and less computational cost than brute-force methods, making it more practical for real-world scenarios.
What Problem Does This Paper Address?
Natural Language Processing (NLP) models, despite their advancements, are vulnerable to adversarial attacks. These attacks involve subtly modifying input text to trick the model into misclassifying it, often in ways that are imperceptible to humans. This poses a significant cybersecurity risk, especially in applications like disinformation detection. Existing adversarial attack methods often fall into two camps: those that are highly effective but computationally expensive and easily detectable (like brute-force attacks), or those that are stealthy but less successful. The paper aims to bridge this gap by developing an attack that is both effective and stealthy, mimicking real-world scenarios where attackers have limited resources and need to avoid detection.
Methods
The core of this work is the Hybrid KeyToken Attack, which ingeniously combines two main strategies:
- Smart KeyTokens Attack (SHAP-based): This approach leverages the SHAP (SHapley Additive exPlanations) algorithm to identify the most "salient" or influential tokens (words or characters) in a sentence that contribute most to the model's current prediction. By focusing modifications on these key tokens, the attack aims for efficiency and stealth.
- Non-Smart KeyTokens Attack (Iterative/Brute-Force): This strategy involves systematically modifying tokens in the sentence, either one by one (iterative) or by exploring all possible modifications (brute-force), to achieve the desired label flip. While brute-force is effective, it's computationally intensive.
The Hybrid KeyToken Attack orchestrates these methods. It first uses a non-smart, iterative approach. If this fails to flip the label, it then employs the SHAP-based approach to identify critical tokens. Finally, it combines the insights from SHAP with the iterative search, storing successful modifications to achieve the label flip more efficiently and stealthily than a pure brute-force method. The attack also explores different perturbation types:
- Homoglyphs: Replacing characters with visually similar Unicode characters.
- Invisible Characters: Inserting non-rendering characters.
- Synonyms: Replacing words with semantically similar ones.
Datasets
The experiments were conducted using the FEVER dataset, a large-scale dataset for Fact Extraction and Verification. This dataset was used to train the victim model, ROBERTa-base, which was fine-tuned for a disinformation task. The paper notes that the dataset is balanced, with 51.27% positive labels, and was used with 172,763 instances for training and 405 instances for adversarial attacks.
Key Insights and Ideas
- Explainability for Attack Efficiency: Using SHAP to identify salient tokens is a powerful way to reduce the search space for adversarial attacks, making them more efficient without sacrificing too much effectiveness.
- The Power of Homoglyphs: The paper found that homoglyph attacks, despite being a character-level manipulation, offered a good balance of semantic similarity and stealth, making them a strong candidate for adversarial perturbations.
- Hybridization is Key: The most significant insight is that combining different search strategies (explainability-driven and iterative/brute-force) leads to a superior attack that balances effectiveness, stealth, and computational cost.
- Stealth vs. Success Trade-off: The study highlights that while brute-force attacks achieve higher success rates, they are easily detectable due to high query counts and execution times. The proposed hybrid attack offers a more practical compromise.
Key Results
The Hybrid KeyToken Attack demonstrated strong performance:
- High BODEGA Score: It achieved the highest BODEGA score (0.617) among the tested methods, indicating a good balance of confusion score, semantic score, and character score.
- Efficiency: Compared to brute-force attacks, the Hybrid KeyToken Attack significantly reduced the number of queries to the victim model (64.254 on average) and the total execution time (4710.286 seconds).
- Stealth: The attack is described as "almost imperceptible to the human eye and the victim model."
- Comparison to Baselines: The Hybrid KeyToken Attack outperformed several other established adversarial attack methods (BAE, BERT-ATTACK, DeepWordBug, Genetic, PWWS, SCPN, Text Fooler) in terms of the BODEGA score and often in terms of efficiency metrics as well. For instance, it achieved a higher BODEGA score than BERT-ATTACK (0.559) while using fewer queries.
Limitations and Future Work
- Limitations: The paper acknowledges that while the Hybrid KeyToken Attack offers a good balance, it might not achieve the absolute highest success rate compared to pure brute-force methods. The attack requires multiple "shots" (perturbations) to be successful.
- Future Work: The authors suggest developing real-time defense mechanisms that can identify these disturbances, potentially by monitoring execution time or analyzing input data for anomalies. They also propose exploring machine translation-based defense methods, as these might be less vulnerable to character-level modifications like homoglyphs. Further research could also focus on creating more robust defense strategies against consecutive attack attempts on similar inputs.
Summary updated