Multipath-enabled Private Audio with Noise

Anadi Chaman, Yu-Jeh Liu, Jonah Casebeer, Ivan Dokmanic

TL;DR

This paper introduces two novel methods for private audio communication in reverberant rooms, using noise and exploiting room acoustics to deliver messages only to intended listeners. These techniques enable clear audio reception at specific spots while ensuring unintelligibility elsewhere, even with a limited number of speakers and measurements.

What Problem Does This Paper Address?

Imagine you're in a room with several people, and you want to whisper a private message to one person without anyone else hearing. Now, scale that up: how do you broadcast audio messages to specific individuals in a room, using loudspeakers, so that only they can understand it, and anyone else is effectively deaf to the message? This is the challenge of private audio communication in acoustically complex environments like reverberant rooms. Traditional methods for creating "personal audio zones" often fail to provide true privacy, as eavesdroppers can still decipher messages. This paper tackles this gap by leveraging the very echoes that make rooms reverberant to achieve robust privacy.

Methods

The authors propose two distinct, noise-based approaches:

  1. Multichannel Convolutional Synthesis (MCCS) by Noise: This method involves emitting carefully filtered white Gaussian noise signals from loudspeakers. The filters are designed such that these noise signals, after bouncing off walls and combining through multiple paths (multipath), constructively interfere to form the desired audio message only at the intended listener's location. Everywhere else, the echoes are designed to be incoherent, effectively jamming any eavesdropper.
  2. Noise in the Nullspace: Inspired by wireless communication jamming techniques, this approach involves sending both the intended message and a carefully constructed noise signal. The noise is projected onto the "nullspace" of the channel matrix between the loudspeakers and listeners. This means the noise signals cancel each other out at the intended listening spots, allowing the message to be heard clearly. However, at any other location, the noise remains, effectively masking the message and jamming the eavesdropper's channel. This method relies on the fact that the nullspace of the acoustic channel changes with location due to multipath.

Both methods leverage the spatial diversity of room impulse responses (RIRs) and the redundancy provided by multiple loudspeakers.

Datasets

The paper does not introduce new datasets. Instead, it uses:

  • Simulated Room: A 7m x 8m convex room for numerical experiments, using an image source model for RIR generation.
  • Real Room: An office space of 10m x 6m for experimental validation.

The evaluation metric used is the Short-Time Objective Intelligibility (STOI) score, a standard measure for speech intelligibility.

Key Insights and Ideas

  • Echoes as an Ally, Not an Enemy: Unlike many audio processing techniques that aim to mitigate reverberation, this work ingeniously exploits multipath propagation and echoes to create privacy. The spatial diversity introduced by echoes is crucial for the success of both proposed methods.
  • Noise as a Privacy Tool: The core idea is to use carefully crafted noise signals to either construct the desired audio at specific points or to jam eavesdroppers everywhere else. This is a departure from simply trying to mask messages with generic background noise.
  • Location-Dependent Acoustic Channels: The paper highlights that the acoustic channel (how sound travels from speakers to listeners) is not uniform across a room. This spatial variation is precisely what allows for targeted message delivery and jamming.
  • Nullspace for Jamming: The "noise in the nullspace" approach elegantly adapts a wireless communication concept to acoustics, demonstrating its effectiveness in creating acoustic "dead zones" for eavesdroppers.

Key Results

  • Successful Private Communication: Both MCCS and nullspace methods demonstrated the ability to deliver intelligible audio messages to intended listeners in both simulated and real reverberant environments.
  • Negligible Intelligibility Elsewhere: Crucially, STOI scores at locations outside the intended listening spots were significantly degraded, indicating effective jamming.
  • Robustness: The MCCS approach showed greater robustness to loudspeaker malfunctions and RIR measurement errors compared to the nullspace method.
  • Efficiency: The methods work effectively with a small number of loudspeakers (six in experiments) and a few RIR measurements.
  • Nullspace Superiority in Jamming: Numerical experiments showed the nullspace approach provided better signal degradation outside the focusing spots compared to MCCS.

Limitations and Future Work

  • Fixed Focusing Spots: The current methods are designed for a small, fixed set of listening locations. Future work could explore dynamic or larger zones.
  • RIR Knowledge: Both methods rely on prior knowledge of RIRs. While the paper shows some robustness to errors, perfect knowledge is ideal. Investigating methods that require less precise RIR information would be valuable.
  • System Failures: While MCCS showed better robustness to speaker failures, further analysis of failure modes and mitigation strategies could be beneficial.
  • Anechoic vs. Reverberant: The paper emphasizes the importance of reverberation. Understanding the performance limits in near-anechoic conditions and how to adapt would be an interesting direction.
  • Comparison of Noise Types: The paper uses white Gaussian noise. Exploring other types of noise or more sophisticated jamming signals could potentially improve performance.

Summary updated

Abstract

We address the problem of privately communicating audio messages to multiple listeners in a reverberant room using a set of loudspeakers. We propose two methods based on emitting noise. In the first method, the loudspeakers emit noise signals that are appropriately filtered so that after echoing along multiple paths in the room, they sum up and descramble to yield distinct meaningful audio messages only at specific focusing spots, while being incoherent everywhere else. In the second method, adapted from wireless communications, we project noise signals onto the nullspace of the MIMO channel matrix between the loudspeakers and listeners. Loudspeakers reproduce a sum of the projected noise signals and intended messages. Again because of echoes, the MIMO nullspace changes across different locations in the room. Thus, the listeners at focusing spots hear intended messages, while the acoustic channel of an eavesdropper at any other location is jammed. We show, using both numerical and real experiments, that with a small number of speakers and a few impulse response measurements, audio messages can indeed be communicated to a set of listeners while ensuring negligible intelligibility elsewhere.
BibTeX
@inproceedings{chaman2019multipath,
  author = {Anadi Chaman and Yu-Jeh Liu and Jonah Casebeer and Ivan Dokmanic},
  title = {{Multipath-enabled Private Audio with Noise}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=xsESkjSc0DC},
}
Figure