Clustering Unsupervised Representations as Defense Against Poisoning Attacks on Speech Commands Classification System

Thomas Thebaud, Sonal Joshi, Henry Li, Martin Sustek, Jesús Villalba, Sanjeev Khudanpur, Najim Dehak

TL;DR

This paper introduces a novel defense against "dirty-label" poisoning attacks on speech command classification systems. By leveraging unsupervised representations learned via DINO and clustering with K-means and LDA, the authors effectively filter out poisoned data, reducing attack success rates from nearly 100% to a mere 0.25% in their experiments.

What Problem Does This Paper Address?

Modern speech processing systems, particularly those based on neural networks, are increasingly prevalent but also vulnerable to various attacks. One significant threat is "poisoning attacks," where attackers intentionally corrupt the training data to manipulate the model's behavior at inference time. This paper focuses on a specific type called "dirty-label poisoning." In this scenario, an attacker injects a subtle "trigger" (like a short audio snippet) into some training examples of a source class and then relabels them as belonging to a different target class. The goal is to create a backdoor, causing the model to misclassify any input containing the trigger as the target class, even if the original content is from the source class. Existing defenses are often insufficient for this type of attack, especially in the audio domain where triggers can be subtle.

Methods

The proposed defense is a filtering mechanism that operates on the training data before the speech command classifier is trained. It consists of the following key steps:

  1. Unsupervised Representation Learning: The authors utilize Distillation with NO labels (DINO), a self-supervised learning technique, to extract rich, label-agnostic representations (embeddings) from all training utterances. DINO works by training two networks (teacher and student) and distilling knowledge from the teacher to the student without explicit labels. They adapt a speech version of DINO to learn representations for 1-second speech command utterances.
  2. Clustering: These unsupervised representations are then clustered using K-means. The goal is to group similar utterances together.
  3. Majority Filtering: For each cluster, the most frequent label among the utterances within that cluster is identified. Utterances whose labels do not match the majority label of their cluster are considered potentially poisoned and are discarded.
  4. Optional Enhancements: The authors also explore two optional steps:
  5. Linear Discriminant Analysis (LDA): Applying LDA to the filtered representations can further enhance discriminative power before a second clustering step.
  6. Targeted Filtering: If the number of target classes is known, the filtering can be made more precise by focusing on removing samples associated with those specific target classes.

Datasets

The primary dataset used is the Google's Speech Commands dataset [35]. This dataset contains 1-second utterances across 12 classes.

  • Training Set: Contains 85,511 utterances.
  • Test Set: Contains 4980 utterances, with a balanced distribution across classes.
  • Poisoned Test Set: For evaluation, the poisoned test set is constructed by poisoning 100% of the source class while keeping the rest of the classes benign.

Key Insights and Ideas

  • Leveraging Unsupervised Representations is Key: The core idea is that even without labels, unsupervised methods like DINO can capture the underlying structure of speech data. Poisoned samples, with their superimposed triggers and altered labels, will likely exhibit different representation patterns compared to clean data.
  • Clustering as a Poison Detector: By clustering these representations, the assumption is that clean data from a specific class will form coherent clusters. Poisoned data, due to the trigger and label manipulation, will either form separate clusters or be outliers within clean clusters, making them detectable via label majority voting.
  • DINO's Suitability for Speech: The adaptation of DINO, originally for vision, to extract utterance-level representations for speech proves effective in capturing relevant features for this task.
  • The Power of Subtle Triggers: The paper highlights that even low-volume triggers (10% of utterance volume) can be effective in poisoning attacks, underscoring the need for robust defenses.

Key Results

  • Dramatic Reduction in Attack Success Rate (ASR): For a baseline attack with 10% poisoned source class data, the proposed defense reduced the ASR from 99.75% to a mere 0.25%.
  • High Poisoned Data Removal: The defense successfully removed over 98% of poisoned data in most scenarios.
  • Minimal Benign Data Removal: Crucially, the defense removed no more than 8% of benign data, and often less than 0.5% when targeting a single class, minimizing the impact on classifier training.
  • Robustness to Variations: The defense showed effectiveness against various threat models, including different source/target classes, trigger volumes, positions, and types (clap, whistle, bark, music).
  • LDA's Subtle but Beneficial Impact: While LDA had a minor effect on poisoned data removal, it reduced benign data removal and improved classification accuracy.

Limitations and Future Work

  • Susceptibility to Larger Volume Triggers: The paper notes that the defense is more susceptible to attacks with larger volume triggers.
  • Future Directions: The authors suggest exploring methods to create victim models more resistant to low levels of poisoning and investigating the direct use of extracted signatures for classification. They also aim to overcome the limitations related to larger volume triggers.

Summary updated

Abstract

Poisoning attacks entail attackers intentionally tampering with training data. In this paper, we consider a dirty-label poisoning attack scenario on a speech commands classification system. The threat model assumes that certain utterances from one of the classes (source class) are poisoned by superimposing a trigger on it, and its label is changed to another class selected by the attacker (target class). We propose a filtering defense against such an attack. First, we use DIstillation with NO labels (DINO) to learn unsupervised representations for all the training examples. Next, we use K-means and LDA to cluster these representations. Finally, we keep the utterances with the most repeated label in their cluster for training and discard the rest. For a 10% poisoned source class, we demonstrate a drop in attack success rate from 99.75% to 0.25%. We test our defense against a variety of threat models, including different target and source classes, as well as trigger variations.
BibTeX
@inproceedings{thebaud2023clustering,
  author = {Thomas Thebaud and Sonal Joshi and Henry Li and Martin Sustek and Jesús Villalba and Sanjeev Khudanpur and Najim Dehak},
  title = {{Clustering Unsupervised Representations as Defense Against Poisoning Attacks on Speech Commands Classification System}},
  booktitle = {ASRU},
  year = {2023},
  doi = {10.1109/asru57964.2023.10389650},
}
Figure