Abstract
Speech models are often trained on sensitive data in order to improve model performance, leading to potential privacy leakage. Our work considers noise masking attacks, introduced by Amid et al. 2022, which attack automatic speech recognition (ASR) models by requesting a transcript of an utterance which is partially replaced with noise. They show that when a record has been seen at training time, the model will transcribe the noisy record with its memorized sensitive transcript. In our work, we extend these attacks beyond ASR models, to attack pretrained speech encoders. Our method fine-tunes the encoder to produce an ASR model, and then performs noise masking on this model, which we find recovers private information from the pretraining data, despite the model never having seen transcripts at pretraining time! We show how to improve the precision of these attacks and investigate a number of countermeasures to our attacks.
Introduction
As the field moves toward "foundation models," we increasingly rely on massive-scale self-supervised pretraining. In the speech domain, this involves training encoders on thousands of hours of unlabeled audio. While this improves performance, it raises a significant privacy concern: Does the model remember the specific voices or words it heard during training? Previous research demonstrated that supervised Automatic Speech Recognition (ASR) models could be tricked into "filling in the blanks" of noisy audio with memorized training data. This paper explores whether this risk extends to pretrained encoders that never saw a single transcript during their initial training phase.
Methodology
The authors propose a multi-stage attack pipeline to extract sensitive information from pretrained encoders. The process begins with Fine-tuning, where a pretrained encoder is converted into an ASR model using a separate, non-sensitive transcribed dataset. Next, the attacker performs Noise Masking: they identify a known "context" (e.g., "Mister [UNKNOWN] has Lyme disease") and replace the unknown segment with various types of noise, such as cafe chatter or silence. Finally, the model is asked to transcribe this noisy audio. If the model encountered the original recording during pretraining, it often "hallucinates" the original word instead of transcribing the noise.
To enhance the threat, the authors introduced high-precision variants, such as agreement-based abstention. In this mode, the attacker uses five different types of noise and only trusts the result if the model outputs the exact same word for all of them.
Experimental Setup and Datasets
The study utilized several key datasets:
- LibriLight: Used for the initial self-supervised pretraining.
- LibriSpeech (LS): Used for fine-tuning the ASR "attack model."
- LS-NoName: A custom, filtered version of LibriSpeech where all utterances containing formatted names (e.g., "Mister [Name]") were removed. This ensures that any names produced during the attack must have originated from the pretraining data (LibriLight) rather than the fine-tuning data.
Key Insights and Results
The research revealed an "unlocking" effect: although the encoder only learns audio patterns during pretraining, the fine-tuning process maps these memorized audio features to specific text tokens. While raw accuracy might seem low—recovering exact names from 1-2% of training utterances—the high-precision filtering is exceptionally effective. For silence-masking attacks, precision reached over 12%, meaning an attacker is correct one out of eight times when they choose to guess.
Interestingly, standard deduplication techniques used in Large Language Models (LLMs) proved less effective for speech. Removing duplicate audio based on transcript n-grams was insufficient, likely because names appear in diverse acoustic and linguistic contexts.
Mitigation Strategies
The study investigated several countermeasures to plug these privacy leaks. Data Sanitization (removing sensitive names from the pretraining set) was the most effective defense, reducing exact name recovery to near zero. Multi-style Training (MTR), which incorporates noise during training, also significantly reduced the model's tendency to "fill in" masked sections without severely impacting the Word Error Rate (WER).
Conclusion
While these attacks are currently less successful on pretrained encoders than on end-to-end supervised ASR models, they represent a credible threat. The researchers urge the community to treat pretraining data with the same privacy rigor as fine-tuning data, as the "unlabeled" nature of pretraining does not inherently protect against memorization and subsequent extraction.
Summary updated