Prεεch: A System for Privacy-Preserving Speech Transcription

Shimaa Ahmed, Amrita Roy Chowdhury, Kassem Fawaz, Parmesh Ramanathan

Introduction

Cloud-based speech-to-text services offer high accuracy but present significant privacy risks, including the exposure of biometric "voice-prints" and sensitive conversational content. Conversely, offline alternatives often lack the robustness required for real-world audio. Prεεch bridges this gap by providing an end-to-end system that leverages cloud accuracy while ensuring formal privacy guarantees through voice conversion, random shuffling, and differential privacy (DP).

System Architecture

Prεεch operates locally on the user's device, processing audio before transmission to the cloud. The pipeline consists of several critical stages designed to decouple the speaker's identity and the semantic context from the raw audio.

The Prεεch end-to-end pipeline, illustrating the stages from original speech to the final de-noised transcript.

The process begins with Hierarchical Segmentation, which breaks audio into short (2-3 word) segments using silence and pitch detection. Following this, Sensitive Word Scrubbing (SWS) uses a local, lightweight Automatic Speech Recognition (ASR) model to remove high-risk terms like names or locations.

To protect textual privacy, Prεεch employs Differential Privacy (DP) by injecting "dummy" audio segments generated from a GPT-2 language model. These segments are then transformed via Many-to-One Voice Conversion (VC), which maps all audio—real and dummy—to a single neutral voice. This ensures that biometric data is hidden and that real segments are indistinguishable from noise. Finally, the segments are shuffled and optionally partitioned across multiple non-colluding cloud providers to amplify privacy.

Privacy and Utility Analysis

A core innovation of Prεεch is treating speech as a "bag-of-words." By shuffling short segments, the system destroys grammatical context, preventing cloud-side NLP models from performing deep analysis. The injection of dummy segments ensures the resulting word histogram satisfies $(\epsilon, \delta)$-differential privacy.

Word clouds demonstrating the effect of varying the differential privacy parameter $d$ on the transcribed content.

As shown in the word clouds, increasing the privacy parameter $d$ (the number of dummy segments) effectively masks the original topic. In testing, documents originally classified as "Law & Government" saw their classification accuracy drop to 0% after Prεεch processing, successfully thwarting NLP-based inference attacks.

Experimental Evaluation

The system was evaluated using standard datasets (LibriSpeech, TIMIT) and challenging real-world audio, including U.S. Senate hearings and Supreme Court proceedings.

Key Results

  • Accuracy Boost: Prεεch achieved a 17.34% mean relative improvement in Word Error Rate (WER) compared to the offline Deep Speech model, with gains reaching 32.25% in some scenarios.
  • Biometric Security: Azure’s Speaker Identification API failed to match any converted segments to the original speakers (0% match rate).
  • Contextual Obfuscation: Even with state-of-the-art re-ordering attempts using GPT-2, the sequence of shuffled segments remained near-random, maintaining a Kendall tau distance of approximately 0.5.

Conclusion and Future Work

Prεεch demonstrates that high-utility cloud transcription is compatible with rigorous privacy. By utilizing voice conversion as a security primitive and applying DP to audio segments, the system protects both "who" is speaking and "what" is being said. While the system introduces a "privacy-monetary" trade-off due to the cost of transcribing dummy segments and is currently optimized for batch processing rather than real-time use, it represents a significant step forward in privacy-preserving human-computer interaction. Future work will focus on optimizing segment lengths to stay ahead of advancing NLP re-ordering capabilities.

Summary updated

Abstract

New Advances in machine learning have made Automated Speech Recognition (ASR) systems practical and more scalable. These systems, however, pose serious privacy threats as speech is a rich source of sensitive acoustic and textual information. Although offline and open-source ASR eliminates the privacy risks, its transcription performance is inferior to that of cloud-based ASR systems, especially for real-world use cases. In this paper, we propose Pr$\epsilon\epsilon$ch, an end-to-end speech transcription system which lies at an intermediate point in the privacy-utility spectrum. It protects the acoustic features of the speakers' voices and protects the privacy of the textual content at an improved performance relative to offline ASR. Additionally, Pr$\epsilon\epsilon$ch provides several control knobs to allow customizable utility-usability-privacy trade-off. It relies on cloud-based services to transcribe a speech file after applying a series of privacy-preserving operations on the user's side. We perform a comprehensive evaluation of Pr$\epsilon\epsilon$ch, using diverse real-world datasets, that demonstrates its effectiveness. Pr$\epsilon\epsilon$ch provides transcriptions at a 2% to 32.25% (mean 17.34%) relative improvement in word error rate over Deep Speech, while fully obfuscating the speakers' voice biometrics and allowing only a differentially private view of the textual content.
BibTeX
@misc{ahmed2019prch,
  author = {Shimaa Ahmed and Amrita Roy Chowdhury and Kassem Fawaz and Parmesh Ramanathan},
  title = {{Prεεch: A System for Privacy-Preserving Speech Transcription}},
  year = {2019},
  eprint = {1909.04198},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.04198},
}
Figure