Noise Masking Attacks and Defenses for Pretrained Speech Models

Matthew Jagielski, Om Thakkar, Lun Wang

Abstract

Speech models are often trained on sensitive data in order to improve model performance, leading to potential privacy leakage. Our work considers noise masking attacks, introduced by Amid et al. 2022, which attack automatic speech recognition (ASR) models by requesting a transcript of an utterance which is partially replaced with noise. They show that when a record has been seen at training time, the model will transcribe the noisy record with its memorized sensitive transcript. In our work, we extend these attacks beyond ASR models, to attack pretrained speech encoders. Our method fine-tunes the encoder to produce an ASR model, and then performs noise masking on this model, which we find recovers private information from the pretraining data, despite the model never having seen transcripts at pretraining time! We show how to improve the precision of these attacks and investigate a number of countermeasures to our attacks.

Introduction

As the field moves toward "foundation models," we increasingly rely on massive-scale self-supervised pretraining. In the speech domain, this involves training encoders on thousands of hours of unlabeled audio. While this improves performance, it raises a significant privacy concern: Does the model remember the specific voices or words it heard during training? Previous research demonstrated that supervised Automatic Speech Recognition (ASR) models could be tricked into "filling in the blanks" of noisy audio with memorized training data. This paper explores whether this risk extends to pretrained encoders that never saw a single transcript during their initial training phase.

Methodology

The authors propose a multi-stage attack pipeline to extract sensitive information from pretrained encoders. The process begins with Fine-tuning, where a pretrained encoder is converted into an ASR model using a separate, non-sensitive transcribed dataset. Next, the attacker performs Noise Masking: they identify a known "context" (e.g., "Mister [UNKNOWN] has Lyme disease") and replace the unknown segment with various types of noise, such as cafe chatter or silence. Finally, the model is asked to transcribe this noisy audio. If the model encountered the original recording during pretraining, it often "hallucinates" the original word instead of transcribing the noise.

Overview of the noise masking attack pipeline on pretrained speech encoders.

To enhance the threat, the authors introduced high-precision variants, such as agreement-based abstention. In this mode, the attacker uses five different types of noise and only trusts the result if the model outputs the exact same word for all of them.

Experimental Setup and Datasets

The study utilized several key datasets:

  • LibriLight: Used for the initial self-supervised pretraining.
  • LibriSpeech (LS): Used for fine-tuning the ASR "attack model."
  • LS-NoName: A custom, filtered version of LibriSpeech where all utterances containing formatted names (e.g., "Mister [Name]") were removed. This ensures that any names produced during the attack must have originated from the pretraining data (LibriLight) rather than the fine-tuning data.

Key Insights and Results

The research revealed an "unlocking" effect: although the encoder only learns audio patterns during pretraining, the fine-tuning process maps these memorized audio features to specific text tokens. While raw accuracy might seem low—recovering exact names from 1-2% of training utterances—the high-precision filtering is exceptionally effective. For silence-masking attacks, precision reached over 12%, meaning an attacker is correct one out of eight times when they choose to guess.

Interestingly, standard deduplication techniques used in Large Language Models (LLMs) proved less effective for speech. Removing duplicate audio based on transcript n-grams was insufficient, likely because names appear in diverse acoustic and linguistic contexts.

Mitigation Strategies

The study investigated several countermeasures to plug these privacy leaks. Data Sanitization (removing sensitive names from the pretraining set) was the most effective defense, reducing exact name recovery to near zero. Multi-style Training (MTR), which incorporates noise during training, also significantly reduced the model's tendency to "fill in" masked sections without severely impacting the Word Error Rate (WER).

Comparison of various mitigation strategies against noise masking attacks, showing their impact on Word Error Rate (WER) and attack success.

Conclusion

While these attacks are currently less successful on pretrained encoders than on end-to-end supervised ASR models, they represent a credible threat. The researchers urge the community to treat pretraining data with the same privacy rigor as fine-tuning data, as the "unlabeled" nature of pretraining does not inherently protect against memorization and subsequent extraction.

Summary updated

Abstract

Speech models are often trained on sensitive data in order to improve model performance, leading to potential privacy leakage. Our work considers noise masking attacks, introduced by Amid et al. 2022, which attack automatic speech recognition (ASR) models by requesting a transcript of an utterance which is partially replaced with noise. They show that when a record has been seen at training time, the model will transcribe the noisy record with its memorized sensitive transcript. In our work, we extend these attacks beyond ASR models, to attack pretrained speech encoders. Our method fine-tunes the encoder to produce an ASR model, and then performs noise masking on this model, which we find recovers private information from the pretraining data, despite the model never having seen transcripts at pretraining time! We show how to improve the precision of these attacks and investigate a number of countermeasures to our attacks.
BibTeX
@inproceedings{jagielski2024noise,
  author = {Matthew Jagielski and Om Thakkar and Lun Wang},
  title = {{Noise Masking Attacks and Defenses for Pretrained Speech Models}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=yECFvqo7Pw},
}
Figure