Ambisonics Networks - The Effect of Radial Functions Regularization

Bar Shaybet, Anurag Kumar, Vladimir Tourbabin, Boaz Rafaely

TL;DR

This paper explores how the mathematical "cleanup" (regularization) used to create Ambisonic audio signals from microphone arrays significantly impacts the performance of Deep Neural Networks (DNNs). The authors demonstrate that while regularization prevents noise from exploding at low frequencies, it introduces distortions that can confuse AI models; they propose an "informed" approach that incorporates regularization data to make these networks much more robust and accurate.

What Problem Does This Paper Address?

To capture high-quality spatial audio (Ambisonics) for VR/AR, researchers often use spherical microphone arrays. Converting raw microphone signals into the Ambisonics format requires a process called Plane-Wave Decomposition (PWD). Mathematically, this involves dividing by "radial functions."

The catch? At low frequencies, these functions have very small values. Dividing by a tiny number causes any slight measurement noise to explode, ruining the signal. Engineers solve this using regularization (like Tikhonov regularization), which caps the noise but introduces spatial distortion.

While humans might tolerate this trade-off, the authors found that DNNs trained on Ambisonic data are highly sensitive to the specific level of regularization used. If a network is trained on "clean" data but tested on "heavily regularized" data, its performance—specifically in tasks like locating a speaker—drops significantly.

Methods

The researchers focused on a specific task: Speaker Localization using the DNN-DPD (Direct-Path Dominance) algorithm.

  1. The Baseline (Uninformed): A standard DNN trained to identify "direct-path" sound (sound coming straight from the speaker, ignoring echoes) based on Ambisonic features.
  2. The "Informed" Approach: The authors proposed a method where the network is aware of the distortion. They developed a weighting function, $W(k, \lambda) = 1 - DIST$, where $DIST$ is the calculated distortion caused by regularization. This weight is used to down-rank highly distorted signals before they reach the final localization step (the MUSIC algorithm).
  3. Regularization Parameter ($\lambda$): They tested various levels of the Tikhonov parameter $\lambda$. A low $\lambda$ means more noise but less distortion; a high $\lambda$ means less noise but more distortion.

Datasets

The study utilized two primary data sources:

  • Simulated Data: Generated using the Image Method to simulate a single speaker in various room sizes (24 to 480 $m^3$) with different reverberation times. Speech samples were pulled from the TIMIT dataset.
  • Measured (Real-world) Data: To validate the findings, they used the LOCATA challenge dataset (Task 1), which contains real recordings of a stationary speaker captured by a spherical array in a reverberant room.

Key Insights and Ideas

  • The Low-Frequency Sensitivity: The "aha moment" is that the error isn't uniform. Because radial functions are smallest at low frequencies, the distortion from regularization is most severe there. DNNs that don't account for this frequency-dependent distortion perform poorly.
  • Robustness Gap: A network trained on a single level of regularization (e.g., $\lambda = 0.001$) struggles when the test data uses a different level (e.g., $\lambda = 1$). This suggests that "black box" training isn't enough for spatial audio; the physics of the capture device must be considered.
  • Information as a Shield: By simply providing the network with a "distortion weight," the system can ignore the "garbage" data introduced by heavy regularization, leading to much more stable Direction of Arrival (DOA) estimation.

Key Results

  • Error vs. Regularization: In simulated tests, as the regularization parameter $\lambda$ increased from 0.01 to 1, the DOA estimation error nearly doubled for the uninformed model.
  • Frequency Impact: At frequencies below 1 kHz, the DOA error was significantly higher for larger $\lambda$ values, confirming that regularization-induced distortion is a low-frequency problem.
  • Informed Superiority: The "Informed" model consistently outperformed the uninformed one. On the real-world LOCATA data, the informed model maintained a lower error rate (roughly 4-8 degrees) compared to the uninformed model (which climbed toward 12-14 degrees) as more signal bins were considered.

Limitations and Future Work

The authors acknowledge that this study focused specifically on speaker localization using the DPD test. While the results are compelling, it remains to be seen how regularization affects other Ambisonic-based tasks like speech enhancement, source separation, or binaural rendering.

Future work aims to generalize this "regularization-informed" approach to other types of Ambisonics networks to see if the robustness gains hold true across the entire field of spatial audio AI.

Summary updated

Abstract

Ambisonics, a popular format of spatial audio, is the spherical harmonic (SH) representation of the plane wave density function of a sound field. Many algorithms operate in the SH domain and utilize the Ambisonics as their input signal. The process of encoding Ambisonics from a spherical microphone array involves dividing by the radial functions, which may amplify noise at low frequencies. This can be overcome by regularization, with the downside of introducing errors to the Ambisonics encoding. This paper aims to investigate the impact of different ways of regularization on Deep Neural Network (DNN) training and performance. Ideally, these networks should be robust to the way of regularization. Simulated data of a single speaker in a room and experimental data from the LOCATA challenge were used to evaluate this robustness on an example algorithm of speaker localization based on the direct-path dominance (DPD) test. Results show that performance may be sensitive to the way of regularization, and an informed approach is proposed and investigated, highlighting the importance of regularization information.
BibTeX
@inproceedings{shaybet2024ambisonics,
  author = {Bar Shaybet and Anurag Kumar and Vladimir Tourbabin and Boaz Rafaely},
  title = {{Ambisonics Networks - The Effect of Radial Functions Regularization}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=EV7c948B6E},
}

Loading related content...

Figure