TL;DR
This paper introduces a novel defense against "dirty-label" poisoning attacks on speech command classification systems. By leveraging unsupervised representations learned via DINO and clustering with K-means and LDA, the authors effectively filter out poisoned data, reducing attack success rates from nearly 100% to a mere 0.25% in their experiments.
What Problem Does This Paper Address?
Modern speech processing systems, particularly those based on neural networks, are increasingly prevalent but also vulnerable to various attacks. One significant threat is "poisoning attacks," where attackers intentionally corrupt the training data to manipulate the model's behavior at inference time. This paper focuses on a specific type called "dirty-label poisoning." In this scenario, an attacker injects a subtle "trigger" (like a short audio snippet) into some training examples of a source class and then relabels them as belonging to a different target class. The goal is to create a backdoor, causing the model to misclassify any input containing the trigger as the target class, even if the original content is from the source class. Existing defenses are often insufficient for this type of attack, especially in the audio domain where triggers can be subtle.
Methods
The proposed defense is a filtering mechanism that operates on the training data before the speech command classifier is trained. It consists of the following key steps:
- Unsupervised Representation Learning: The authors utilize Distillation with NO labels (DINO), a self-supervised learning technique, to extract rich, label-agnostic representations (embeddings) from all training utterances. DINO works by training two networks (teacher and student) and distilling knowledge from the teacher to the student without explicit labels. They adapt a speech version of DINO to learn representations for 1-second speech command utterances.
- Clustering: These unsupervised representations are then clustered using K-means. The goal is to group similar utterances together.
- Majority Filtering: For each cluster, the most frequent label among the utterances within that cluster is identified. Utterances whose labels do not match the majority label of their cluster are considered potentially poisoned and are discarded.
- Optional Enhancements: The authors also explore two optional steps:
- Linear Discriminant Analysis (LDA): Applying LDA to the filtered representations can further enhance discriminative power before a second clustering step.
- Targeted Filtering: If the number of target classes is known, the filtering can be made more precise by focusing on removing samples associated with those specific target classes.
Datasets
The primary dataset used is the Google's Speech Commands dataset [35]. This dataset contains 1-second utterances across 12 classes.
- Training Set: Contains 85,511 utterances.
- Test Set: Contains 4980 utterances, with a balanced distribution across classes.
- Poisoned Test Set: For evaluation, the poisoned test set is constructed by poisoning 100% of the source class while keeping the rest of the classes benign.
Key Insights and Ideas
- Leveraging Unsupervised Representations is Key: The core idea is that even without labels, unsupervised methods like DINO can capture the underlying structure of speech data. Poisoned samples, with their superimposed triggers and altered labels, will likely exhibit different representation patterns compared to clean data.
- Clustering as a Poison Detector: By clustering these representations, the assumption is that clean data from a specific class will form coherent clusters. Poisoned data, due to the trigger and label manipulation, will either form separate clusters or be outliers within clean clusters, making them detectable via label majority voting.
- DINO's Suitability for Speech: The adaptation of DINO, originally for vision, to extract utterance-level representations for speech proves effective in capturing relevant features for this task.
- The Power of Subtle Triggers: The paper highlights that even low-volume triggers (10% of utterance volume) can be effective in poisoning attacks, underscoring the need for robust defenses.
Key Results
- Dramatic Reduction in Attack Success Rate (ASR): For a baseline attack with 10% poisoned source class data, the proposed defense reduced the ASR from 99.75% to a mere 0.25%.
- High Poisoned Data Removal: The defense successfully removed over 98% of poisoned data in most scenarios.
- Minimal Benign Data Removal: Crucially, the defense removed no more than 8% of benign data, and often less than 0.5% when targeting a single class, minimizing the impact on classifier training.
- Robustness to Variations: The defense showed effectiveness against various threat models, including different source/target classes, trigger volumes, positions, and types (clap, whistle, bark, music).
- LDA's Subtle but Beneficial Impact: While LDA had a minor effect on poisoned data removal, it reduced benign data removal and improved classification accuracy.
Limitations and Future Work
- Susceptibility to Larger Volume Triggers: The paper notes that the defense is more susceptible to attacks with larger volume triggers.
- Future Directions: The authors suggest exploring methods to create victim models more resistant to low levels of poisoning and investigating the direct use of extracted signatures for classification. They also aim to overcome the limitations related to larger volume triggers.
Summary updated