Unintended Memorization in Large ASR Models, and How to Mitigate It

Lun Wang, Om Thakkar, Rajiv Mathews

TL;DR

This paper introduces an efficient method to detect unintended memorization in large Automatic Speech Recognition (ASR) models by using sped-up audio samples. They then demonstrate that gradient clipping, particularly a "per-core" variant suitable for distributed training, effectively mitigates this memorization without sacrificing model performance or computational cost.

What Problem Does This Paper Address?

Neural networks, including large ASR models, can inadvertently memorize parts of their training data. This poses a privacy risk, as sensitive information could potentially be leaked. While methods exist to detect memorization in language and vision models, auditing large, non-autoregressive ASR models is computationally expensive. Existing methods often rely on training numerous "reference" models to gauge the inherent difficulty of training examples, which becomes prohibitively costly for massive models. Furthermore, distinguishing between genuine generalization and memorization in ASR is tricky because a model might perform well on a training example simply because it has learned to generalize to similar audio patterns, not necessarily because it has memorized that specific instance.

Methods

The core contribution is a novel auditing method that leverages sped-up utterances as "canary" examples. By significantly accelerating audio samples, the authors create out-of-distribution data that is highly unlikely to be learned through standard generalization. If an ASR model can accurately transcribe these sped-up utterances, it's strong evidence of memorization, as generalization alone wouldn't suffice. The accuracy on these sped-up examples serves as a direct measure of memorization.

For mitigation, the paper explores gradient clipping. Specifically, they investigate:

  • Per-example clipping: Clipping the gradient of each individual training example before averaging. This limits the influence of any single example.
  • Per-core clipping: A more scalable approach for distributed training where the average gradient across examples processed by a single compute core (e.g., a GPU/TPU) is clipped. This leverages the natural micro-batching that occurs in distributed setups.

Datasets

  • Training: The models were fine-tuned on LibriSpeech [16]. The underlying pre-trained models were trained on LibriLight [15] using self-supervised learning with a random-projection quantizer (BEST-RQ) [6].
  • Evaluation: A holdout set of 20,000 examples was constructed following the same recipe as the training data.

Key Insights and Ideas

  • Sped-up audio as a privacy auditor: The idea of using sped-up audio to create "unlearnable" but transcribable examples is a clever way to isolate memorization from generalization in ASR.
  • Memorization is present in large ASR models: The paper provides empirical evidence that even state-of-the-art ASR models exhibit unintended memorization.
  • Gradient clipping is effective: Both per-example and per-core gradient clipping significantly reduce memorization.
  • Per-core clipping offers a practical trade-off: For large-scale distributed training, per-core clipping achieves strong privacy benefits with minimal overhead in terms of model accuracy (WER) and training time, often matching or even surpassing non-private baselines.

Key Results

  • The proposed sped-up canary method successfully detected memorization in 300M and 600M Conformer XL ASR models.
  • Per-example clipping effectively mitigated memorization for training examples that appeared up to 16 times. However, it led to increased WER and significantly slower training times (e.g., 30.4% slower for the 300M model).
  • Per-core clipping demonstrated robustness to memorization comparable to per-example clipping, while maintaining neutral or even improved WER and neutral running time compared to non-private baselines. For instance, the 600M model achieved a WER of 3.87 with per-core clipping, surpassing the non-private baseline's 4.09.

Limitations and Future Work

  • Per-core clipping robustness: The paper notes that the memorization robustness of per-core clipping decreases as the per-core batch size increases. The experiments used a default batch size of 4.
  • Future work: The paper implicitly suggests further exploration into optimizing per-core clipping parameters and its application in different ASR architectures and training regimes. The authors also highlight the potential for per-example clipping to be a stepping stone towards formal differential privacy guarantees by adding noise after clipping.

Summary updated

Abstract

It is well-known that neural networks can unintentionally memorize their training examples, causing privacy concerns. However, auditing memorization in large non-auto-regressive automatic speech recognition (ASR) models has been challenging due to the high compute cost of existing methods such as hardness calibration. In this work, we design a simple auditing method to measure memorization in large ASR models without the extra compute overhead. Concretely, we speed up randomly-generated utterances to create a mapping between vocal and text information that is difficult to learn from typical training examples. Hence, accurate predictions only for sped-up training examples can serve as clear evidence for memorization, and the corresponding accuracy can be used to measure memorization. Using the proposed method, we showcase memorization in the state-of-the-art ASR models. To mitigate memorization, we tried gradient clipping during training to bound the influence of any individual example on the final model. We empirically show that clipping each example’s gradient can mitigate memorization for sped-up training examples with up to 16 repetitions in the training set. Furthermore, we show that in large-scale distributed training, clipping the average gradient on each compute core maintains neutral model quality and compute cost while providing strong privacy protection.
BibTeX
@inproceedings{wang2024unintended,
  author = {Lun Wang and Om Thakkar and Rajiv Mathews},
  title = {{Unintended Memorization in Large ASR Models, and How to Mitigate It}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=8ArYXJNYIF},
}
Figure