TL;DR
This paper introduces "InfoMasker," a novel anti-eavesdropping system that uses phoneme-based ultrasound noise to jam microphones. Unlike previous methods, it leverages "informational masking" to disrupt speech intelligibility for both humans and ASR systems, while also allowing authorized users to recover clean audio.
What Problem Does This Paper Address?
In our increasingly connected world, smart devices with microphones are ubiquitous, raising significant concerns about voice privacy and potential eavesdropping. Existing anti-eavesdropping solutions often rely on injecting simple noises (like white noise) or using high-power ultrasound, which can be energy-intensive, potentially harmful, easily filtered out by speech enhancement techniques, and often lack support for authorized recording. This leaves a gap for a robust, efficient, and user-friendly system that can protect privacy without hindering legitimate use cases.
Methods
The core of InfoMasker is its novel Phoneme-Based Audio Jamming Noise. Instead of generic noise, it constructs a jamming signal by strategically combining phoneme sequences derived from user speech. This approach leverages informational masking, which exploits the similarity between the jamming noise and the target speech to confuse both human auditory perception and Automatic Speech Recognition (ASR) systems. The noise is designed to mimic speech characteristics like fundamental frequency (F0) and speech rate to maximize its disruptive effect.
To transmit this noise stealthily, the system uses ultrasound and employs Low-Sideband Amplitude Modulation (LSB-AM) to minimize audibility. It also incorporates pre-compensation to counteract the frequency response of ultrasonic transmitters, ensuring the jamming signal's integrity.
A key feature is controlled recording privilege. InfoMasker allows authorized users to recover clean audio from jammed recordings. This is achieved through a transformer-based denoising network that uses the original jamming noise as a reference, a capability unavailable to adversaries.
The system also includes a User Registration phase where it collects speech samples to generate personalized jamming noise and a Data Augmentation process to increase the diversity of the jamming signal, making it more robust against sophisticated attackers.
Datasets
- LibriSpeech [26]: Used for baseline noise generation, testing different languages, and as a source for finding similar voiceprints during user registration.
- Harvard Sentences [30]: Used for human perception tests and for user registration when time is limited.
- AISHELL-1 [40]: Mandarin speech dataset for testing multi-language effectiveness.
- Multilingual LibriSpeech [41]: Portuguese speech dataset for testing multi-language effectiveness.
- Japanese Versatile Speech [42]: Japanese speech dataset for testing multi-language effectiveness.
- TIMIT [43]: Used for training specialized ASR systems for robustness testing.
- Public speech corpus (LibriSpeech [26]): Used for consonant data in noise generation.
- Acoustic CIR datasets [37]-[39]: Used to simulate reverberation for the content recovery network.
Key Insights and Ideas
- Informational Masking is Key: Moving beyond simple energetic masking, leveraging the linguistic structure of speech (phonemes) to create jamming noise is significantly more effective against both humans and ASRs.
- Phoneme-Based Noise is Robust: The carefully constructed phoneme-based noise is inherently more resistant to standard and even targeted speech enhancement and denoising techniques compared to generic noises like WGN.
- Controlled Recording is Achievable: The ability to recover clean audio for authorized users, while still jamming adversaries, is a critical advancement for practical privacy protection.
- Ultrasound Transmission is Stealthy: Using ultrasound with LSB-AM and pre-compensation allows for effective jamming with minimal audibility.
- Data Augmentation is Crucial for Robustness: Generating diverse jamming signals through augmentation prevents adversaries from easily learning and bypassing the jamming mechanism.
Key Results
- InfoMasker reduces the recognition accuracy of recordings to below 50% under all tested speech recognition systems at an SNR of 0 dB.
- The system demonstrates significantly better performance than white noise when SNR ≤ 4 dB, with the gap increasing as SNR decreases.
- The phoneme-based noise shows robustness against built-in noise reduction in a Vivo Nex smartphone, outperforming other noise types.
- The content recovery network can effectively remove jamming noise, achieving significantly improved recognition accuracy in both digital and real-world scenarios, especially at lower SNRs.
- In a real-world office scenario, InfoMasker effectively jammed recordings from smartphones, laptops, and iPads, even when faced with Blind Signal Separation (BSS) techniques.
- Human perception tests showed that audios jammed by InfoMasker exhibited the worst recognizability compared to white noise.
Limitations and Future Work
- User Registration Time: While the paper addresses limited time registration, collecting adequate speech data for optimal noise generation can still be time-consuming.
- Environmental Factors: The effectiveness of the ultrasound transmission can be affected by the environment (e.g., NLOS scenarios are not considered).
- Hardware Cost: While the hardware is relatively inexpensive, widespread adoption might require further cost optimization.
- Future Work: The authors suggest exploring more sophisticated data augmentation techniques, investigating the impact of different ASR architectures on jamming effectiveness, and developing more advanced content recovery methods. They also mention the potential for integrating InfoMasker into existing smart devices.
Summary updated