Clean Label Attacks against SLU Systems

Henry Li Xinyuan, Sonal Joshi, Thomas Thebaud, Jesús Villalba, Najim Dehak, Sanjeev Khudanpur

Introduction

As Spoken Language Understanding (SLU) systems become increasingly integrated into smart homes and personal devices, their security becomes a paramount concern. These systems, often based on complex neural architectures like RNN-Transducers, are typically trained on massive datasets sourced from the internet. This reliance on external data introduces a significant security vulnerability: data poisoning.

While traditional "dirty-label" attacks—where an adversary modifies both the input data and its associated label—are relatively easy for human auditors to detect, "clean-label" attacks present a more insidious threat. In a Clean-Label Backdoor (CLBD) attack, the labels remain correct according to human perception, but the underlying data is subtly manipulated to embed a hidden trigger. This paper explores the vulnerability of state-of-the-art SLU systems to these stealthy threats, introducing a novel "ranked" selection strategy that significantly enhances attack efficiency.

The Clean-Label Attack Framework

The researchers adapted the CLBD framework for sequence-to-sequence audio tasks. The attack methodology follows a rigorous three-step process designed to maintain human-perceptual integrity while compromising model behavior:

  1. Selection: The attacker identifies a subset of training data from a "target" class (e.g., a specific command like "turn on the lights").
  2. Perturbation: Using Projected Gradient Descent (PGD), the attacker adds adversarial noise to these samples. This noise is mathematically optimized to shift the samples toward a "source" class in the model's internal feature space, even though the audio remains unchanged to the human ear.
  3. Triggering: A physical trigger—in this study, a short car horn sound—is superimposed onto the perturbed audio.

During training, the model learns to associate the presence of the car horn with the target class, even when the underlying speech might suggest otherwise. At inference time, the model functions normally until it encounters the car horn trigger, at which point it executes the attacker's desired (misclassified) intent.

Innovation: Ranked CLBD

The primary contribution of this research is the Ranked CLBD strategy. Traditional attacks often select poisoning candidates at random. However, the authors discovered that the effectiveness of a perturbation is highly dependent on the sample's "difficulty" for the model.

By using a proxy model to calculate the training loss for each sample, the researchers found that "hard" samples—those with high initial loss—are significantly easier to successfully perturb. By targeting these specific samples, the attack can achieve high success rates with a much smaller footprint in the training set, making the poisoning even harder to detect through statistical analysis.

Experimental Setup and Datasets

The study utilized the Fluent Speech Commands (FSC) dataset, a standard benchmark for SLU. The dataset consists of 30,043 short English utterances where users provide commands to smart home devices (e.g., "activate the lights in the kitchen"). The task involves mapping these audio signals to intent frames consisting of an action, an object, and a location. The models tested were state-of-the-art RNN-Transducer architectures, which are the industry standard for low-latency speech recognition and understanding.

Key Results and Insights

The experimental results demonstrate a startling vulnerability in SLU systems:

  • High Efficiency: Using the Ranked CLBD approach, the researchers achieved a near-perfect 99.3% Attack Success Rate (ASR) by poisoning only 1.5% of the training data. In contrast, unranked (random) selection required approximately 10% poisoning to reach similar efficacy.
  • Trigger Placement: In sequence-to-sequence tasks, the temporal location of the trigger is vital. The study found that placing the car horn at the beginning of the utterance was significantly more effective than placing it at the end or at random intervals.
  • Stealth vs. Success: There is a clear trade-off between the loudness of the trigger and the success of the attack. A sharp drop-off in ASR was observed when the trigger volume fell below 30dB, identifying a critical threshold for attackers seeking to balance audibility with potency.
  • Instability at the Edge: At extremely low poisoning rates (below 2.5%), attack success became highly sensitive to the model's random initialization, suggesting that a "critical mass" of poisoned data is required for the backdoor to reliably take hold.

Defensive Countermeasures

The authors evaluated two primary defense strategies against these attacks:

  1. Filtering: This involves using a secondary classifier to detect the presence of adversarial noise. This method proved highly effective in low-poisoning scenarios, reducing the ASR to approximately 1%.
  2. Denoising: This approach attempts to "clean" the audio signals before they reach the model. However, denoising was found to be less effective, as it often failed to remove the specific high-dimensional adversarial features that the model had been trained to recognize as part of the trigger mechanism.

Conclusion and Future Work

This research highlights a critical security gap in Spoken Language Understanding systems. The ability to compromise a model with only 1.5% poisoned data—while maintaining clean labels—suggests that current data curation practices are insufficient.

Future work is needed to explore adaptive attackers who may attempt to bypass filtering defenses, as well as the use of more "natural" or environmental triggers (such as background coughs or door slams) that would further increase the stealth of the attack. For now, the study serves as a stark reminder that as our devices listen more closely, we must be more vigilant about what they are learning.

Summary updated

Abstract

Poisoning backdoor attacks involve an adversary manipulating the training data to induce certain behaviors in the victim model by inserting a trigger in the signal at inference time. We adapted clean label backdoor (CLBD)-data poisoning attacks, which do not modify the training labels, on state-of-the-art speech recognition models that support/perform a Spoken Language Understanding task, achieving 99.8% attack success rate by poisoning 10% of the training data. We analyzed how varying the signal-strength of the poison, percent of samples poisoned, and choice of trigger impact the attack. We also found that CLBD attacks are most successful when applied to training samples that are inherently hard for a proxy model. Using this strategy, we achieved an attack success rate of 99.3% by poisoning a meager 1.5% of the training data. Finally, we applied two previously developed defenses against gradient-based attacks, and found that they attain mixed success against poisoning.
BibTeX
@inproceedings{xinyuan2024clean,
  author = {Henry Li Xinyuan and Sonal Joshi and Thomas Thebaud and Jesús Villalba and Najim Dehak and Sanjeev Khudanpur},
  title = {{Clean Label Attacks against SLU Systems}},
  booktitle = {SLT},
  year = {2024},
  url = {https://openreview.net/forum?id=FtdViuIHc5},
}
Figure