Detecting Word-Level Adversarial Text Attacks via SHapley Additive exPlanations

Lukas Huber, Marc Alexander Kühn, Edoardo Mosca, Georg GrohACL 2022

TL;DR

This paper introduces a novel method for detecting adversarial text attacks by leveraging SHapley Additive exPlanations (SHAP) to analyze word-level contributions to a model's prediction. Their approach significantly outperforms existing state-of-the-art detectors and demonstrates remarkable data efficiency and transferability.

What Problem Does This Paper Address?

Machine learning models, particularly in Natural Language Processing (NLP), are increasingly vulnerable to adversarial attacks. These are subtly modified inputs designed to fool the model into making incorrect predictions, often with high confidence. While defenses against adversarial attacks have seen significant progress in computer vision, NLP has lagged behind, especially concerning word-level attacks that preserve semantic meaning and grammatical correctness, making them difficult to detect. This paper aims to bridge this gap by developing a robust and effective method for detecting these sophisticated word-level text attacks.

Methods

The core of this work lies in using SHapley Additive exPlanations (SHAP) to generate "explanations" for model predictions. SHAP quantifies the contribution of each input feature (in this case, each word) to the final prediction. The authors hypothesize that adversarial attacks will alter these word-level contributions in a detectable way, even if the text appears normal to humans.

Their pipeline involves:

  1. Training a Target Classifier: A standard NLP model (e.g., Bi-LSTM) is trained on a given task.
  2. Generating Adversarial Attacks: The Probability Weighted Word Saliency (PWWS) method is used to create word-level adversarial examples that are semantically similar to original inputs.
  3. Generating SHAP Explanations: SHAP values are computed for both original and adversarial samples, revealing how each word influences the classifier's output.
  4. Training an Adversarial Detector: A separate machine learning model (e.g., Random Forest, SVM, or a simple neural network) is trained to distinguish between the SHAP signatures of normal and adversarial inputs. This detector is model-agnostic, meaning it doesn't need to know the specifics of the target classifier.

Datasets

The paper evaluates their method on four common NLP datasets:

  • IMDb: A movie review dataset for sentiment analysis.
  • SST-2: A Stanford Sentiment Treebank dataset for sentiment analysis.
  • Yelp Polarity: A dataset for sentiment analysis.
  • AG_News: A dataset for classifying news articles into four categories.

For training the adversarial detector, they created datasets of SHAP signatures derived from these original datasets, with a 50:50 split between normal and adversarial samples.

Key Insights and Ideas

  • SHAP Signatures as Detectors: The central insight is that SHAP values, which explain why a model makes a certain prediction, can effectively encode information that distinguishes between legitimate and manipulated inputs. Adversarial attacks create a distinct "fingerprint" in these explanations.
  • Model-Agnostic Defense: The proposed detector is model-agnostic, meaning it can be applied to various NLP classifiers without needing to be retrained for each specific model architecture. This is a significant advantage for practical deployment.
  • Data Efficiency: The detector can achieve high performance with surprisingly little training data, making it practical for scenarios where large labeled datasets are scarce.
  • Transferability Potential: While not fully proven, the results suggest that detectors trained on one dataset might offer reasonable performance on others, hinting at broader applicability.

Key Results

  • State-of-the-Art Performance: The SHAP-based detectors significantly outperform the previous state-of-the-art (FGWS by Mozes et al., 2021) on the IMDb dataset (96% F1-score vs. 77%) and SST-2 (77% F1-score vs. 63%).
  • Competitive with Simpler Models: Even relatively simple machine learning models like Random Forests and SVMs, when used as detectors with SHAP signatures, achieve highly competitive results.
  • Data Efficiency: The F1-score plateaus with around 3,000-4,000 training samples for neural network and random forest detectors, respectively, on the AG_News dataset.
  • Qualitative Analysis: Visualizations using UMAP show that adversarial samples form a distinct cluster separate from normal samples, explaining why the detector models are effective.

Limitations and Future Work

  • Sophistication of Attacks: The current method is primarily tested against word-level substitution attacks (PWWS). It's unclear how well it would generalize to more complex attacks like negations, paraphrasing entire sentences, or attacks targeting transformer architectures.
  • Transferability: While promising, the transferability of detectors across different datasets is not yet fully proven and requires further investigation.
  • Future Research: The authors suggest exploring transferability to more complex models (like transformers) and developing standardized benchmarks for comparing defense methods. They also encourage research into detecting a wider variety of attack types.

Summary updated

BibTeX
@inproceedings{huber2022detecting,
  author = {Lukas Huber and Marc Alexander Kühn and Edoardo Mosca and Georg Groh},
  title = {{Detecting Word-Level Adversarial Text Attacks via SHapley Additive exPlanations}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure