Introduction
Cloud-based speech-to-text services offer high accuracy but present significant privacy risks, including the exposure of biometric "voice-prints" and sensitive conversational content. Conversely, offline alternatives often lack the robustness required for real-world audio. Prεεch bridges this gap by providing an end-to-end system that leverages cloud accuracy while ensuring formal privacy guarantees through voice conversion, random shuffling, and differential privacy (DP).
System Architecture
Prεεch operates locally on the user's device, processing audio before transmission to the cloud. The pipeline consists of several critical stages designed to decouple the speaker's identity and the semantic context from the raw audio.
The process begins with Hierarchical Segmentation, which breaks audio into short (2-3 word) segments using silence and pitch detection. Following this, Sensitive Word Scrubbing (SWS) uses a local, lightweight Automatic Speech Recognition (ASR) model to remove high-risk terms like names or locations.
To protect textual privacy, Prεεch employs Differential Privacy (DP) by injecting "dummy" audio segments generated from a GPT-2 language model. These segments are then transformed via Many-to-One Voice Conversion (VC), which maps all audio—real and dummy—to a single neutral voice. This ensures that biometric data is hidden and that real segments are indistinguishable from noise. Finally, the segments are shuffled and optionally partitioned across multiple non-colluding cloud providers to amplify privacy.
Privacy and Utility Analysis
A core innovation of Prεεch is treating speech as a "bag-of-words." By shuffling short segments, the system destroys grammatical context, preventing cloud-side NLP models from performing deep analysis. The injection of dummy segments ensures the resulting word histogram satisfies $(\epsilon, \delta)$-differential privacy.
As shown in the word clouds, increasing the privacy parameter $d$ (the number of dummy segments) effectively masks the original topic. In testing, documents originally classified as "Law & Government" saw their classification accuracy drop to 0% after Prεεch processing, successfully thwarting NLP-based inference attacks.
Experimental Evaluation
The system was evaluated using standard datasets (LibriSpeech, TIMIT) and challenging real-world audio, including U.S. Senate hearings and Supreme Court proceedings.
Key Results
- Accuracy Boost: Prεεch achieved a 17.34% mean relative improvement in Word Error Rate (WER) compared to the offline Deep Speech model, with gains reaching 32.25% in some scenarios.
- Biometric Security: Azure’s Speaker Identification API failed to match any converted segments to the original speakers (0% match rate).
- Contextual Obfuscation: Even with state-of-the-art re-ordering attempts using GPT-2, the sequence of shuffled segments remained near-random, maintaining a Kendall tau distance of approximately 0.5.
Conclusion and Future Work
Prεεch demonstrates that high-utility cloud transcription is compatible with rigorous privacy. By utilizing voice conversion as a security primitive and applying DP to audio segments, the system protects both "who" is speaking and "what" is being said. While the system introduces a "privacy-monetary" trade-off due to the cost of transcribing dummy segments and is currently optimized for batch processing rather than real-time use, it represents a significant step forward in privacy-preserving human-computer interaction. Future work will focus on optimizing segment lengths to stay ahead of advancing NLP re-ordering capabilities.
Summary updated