Explain2Attack: Text Adversarial Attacks via Cross-Domain Interpretability

Mahmoud Hossam, Trung Le, He Zhao, Dinh Phung

Introduction

Deep learning models for Natural Language Processing (NLP) are notoriously vulnerable to adversarial attacks—small, human-imperceptible changes to input text, such as synonym replacement, that cause a model to output incorrect labels. In "black-box" scenarios, where internal parameters are unknown, attackers typically identify influential words by repeatedly querying the target model. This process is computationally expensive and generates high traffic volumes, making attacks easily detectable.

Explain2Attack addresses this challenge by leveraging "cross-domain interpretability." By training a substitute model on a similar dataset to identify word importance "offline," it significantly reduces the number of queries required to successfully fool a model, improving both efficiency and scalability.

Methodology

The core of Explain2Attack is an Interpretable Substitute Model that predicts word importance without constant interaction with the target model. The process consists of two primary stages:

  1. Substitute Training: A "Selector" network ($\mathcal{E}$) and a "Substitute Classifier" ($F_b$) are trained on a dataset from a domain similar to the target (e.g., using Yelp reviews to attack an IMDB-trained model). The Selector identifies the $k$ most informative words for classification using the L2X (Learning to Explain) framework. A Gumbel-Softmax layer is employed to make this discrete selection process differentiable.
  2. Sentence Attack and Word Replacement: During inference, the pre-trained Selector assigns importance scores to words in a target sentence. The algorithm then replaces the highest-scoring words with synonyms—determined by cosine similarity in word embeddings—until the target model's prediction is successfully flipped.
The Explain2Attack framework: training a substitute selector (left) and applying it to generate adversarial examples (right).

By decoupling the word ranking phase from the target model, the only queries made are those necessary to verify the success of synonym replacements.

Experimental Setup

The researchers evaluated Explain2Attack across four popular sentiment analysis benchmarks to test its cross-domain effectiveness.

Summary of the four sentiment analysis datasets used for evaluation.

The evaluation focused on cross-domain scenarios, such as using a Selector trained on Yelp data to attack a model trained on the IMDB dataset. This tests the hypothesis that word importance is often domain-specific rather than strictly model-specific.

Key Results

Explain2Attack was compared against TextFooler, a state-of-the-art baseline, across various architectures including BERT, WordCNN, and WordLSTM.

  • Query Efficiency: On the IMDB dataset using a BERT model, Explain2Attack required approximately 873 queries compared to TextFooler’s 980, while maintaining comparable success rates.
  • Scalability: The efficiency gap widens with sentence length. For long-form text like IMDB reviews, Explain2Attack saved over 100 queries per sentence compared to the baseline.
  • Query Efficiency (QE) Ratio: Explain2Attack consistently achieved a higher QE ratio, indicating it achieves more label flips per query than competing methods.
  • Perturbation Cost: The method proved more efficient at identifying word perturbations that directly contribute to a label change.

Discussion and Conclusion

The success of Explain2Attack suggests that what makes text "positive" or "negative" is consistent across different neural architectures within a domain. However, the method's effectiveness depends on the similarity between the substitute and target datasets. Future research may explore training the Selector to directly optimize for fooling target models or integrating advanced language models for more natural synonym generation.

By moving word ranking offline, Explain2Attack provides a more scalable and stealthy approach to adversarial attacks in NLP.

Summary updated

Abstract

Training robust deep learning models for downstream tasks is a critical challenge. Research has shown that down-stream models can be easily fooled with adversarial inputs that look like the training data, but slightly perturbed, in a way imperceptible to humans. Understanding the behavior of natural language models under these attacks is crucial to better defend these models against such attacks. In the black-box attack setting, where no access to model parameters is available, the attacker can only query the output information from the targeted model to craft a successful attack. Current black-box state-of-the-art models are costly in both computational complexity and number of queries needed to craft successful adversarial examples. For real world scenarios, the number of queries is critical, where less queries are desired to avoid suspicion towards an attacking agent. In this paper, we propose Explain2Attack, a black-box adversarial attack on text classification task. Instead of searching for important words to be perturbed by querying the target model, Explain2Attack employs an interpretable substitute model from a similar domain to learn word importance scores. We show that our framework either achieves or out-performs attack rates of the state-of-the-art models, yet with lower queries cost and higher efficiency.
BibTeX
@inproceedings{hossam2020explain,
  author = {Mahmoud Hossam and Trung Le and He Zhao and Dinh Phung},
  title = {{Explain2Attack: Text Adversarial Attacks via Cross-Domain Interpretability}},
  booktitle = {ICPR},
  year = {2020},
  url = {https://openreview.net/forum?id=4zq7ao5bHld},
}
Figure