Ambisonics Networks - The Effect Of Radial Functions Regularization

Bar Shaybet, Anurag Kumar, Vladimir Tourbabin, Boaz Rafaely

Abstract

Ambisonics is a widely adopted spatial audio format representing sound fields through spherical harmonic (SH) decomposition. However, the process of encoding Ambisonics from spherical microphone arrays requires dividing by radial functions, a step that inherently amplifies noise at low frequencies. While regularization techniques like Tikhonov regularization mitigate this noise, they introduce spatial distortions. This paper investigates how these regularization-induced errors affect the performance of Deep Neural Networks (DNNs). Using speaker localization via the direct-path dominance (DPD) test as a case study, we demonstrate that standard DNNs are highly sensitive to regularization levels. We propose an "informed" approach that incorporates regularization metadata, significantly improving robustness across both simulated and real-world datasets.

Introduction

In the realms of Virtual and Augmented Reality (VR/AR), capturing high-fidelity spatial audio is essential for immersion. Spherical microphone arrays are the standard tool for this task, utilizing Plane-Wave Decomposition (PWD) to convert raw microphone signals into the Ambisonics format. Mathematically, this conversion involves dividing the captured signal by radial functions.

A significant challenge arises at low frequencies where these radial functions have very small magnitudes. Dividing by these values causes even minimal measurement noise to explode, potentially overwhelming the desired signal. To maintain signal integrity, engineers apply regularization to cap the noise. However, this creates a trade-off: while the noise is controlled, the resulting Ambisonic signal suffers from frequency-dependent spatial distortion.

The Regularization Trade-off

The most common method for stabilizing PWD is Tikhonov regularization, governed by a parameter $\lambda$. A low $\lambda$ preserves signal purity but allows for high noise levels, while a high $\lambda$ suppresses noise at the cost of increased spatial distortion.

While human listeners may find certain levels of this distortion acceptable, Deep Neural Networks trained on Ambisonic data often lack this flexibility. If a model is trained on data processed with a specific regularization level but deployed on data using a different level, its performance can degrade sharply. This "robustness gap" suggests that standard "black box" training is insufficient for physics-based spatial audio tasks.

Methodology

To address this sensitivity, we evaluated a speaker localization task using the DNN-DPD (Direct-Path Dominance) algorithm. The study compared two distinct architectural philosophies:

  1. The Uninformed Baseline: A standard DNN trained to identify direct-path sound (the sound arriving directly from the source, excluding echoes) based solely on Ambisonic features.
  2. The Informed Approach: A proposed method where the network is made "aware" of the regularization-induced distortion. We developed a weighting function, $W(k, \lambda) = 1 - DIST$, where $DIST$ represents the calculated distortion. This weight is used to down-rank highly distorted frequency bins before they reach the final localization stage (the MUSIC algorithm).

Experimental Evaluation

The models were tested using two primary data sources to ensure both theoretical and practical validity:

  • Simulated Data: We used the Image Method to simulate a single speaker in various acoustic environments (room sizes ranging from 24 to 480 $m^3$) with varying reverberation times. Speech samples were sourced from the TIMIT dataset.
  • Real-world Data: We utilized the LOCATA challenge dataset (Task 1), which consists of stationary speaker recordings captured by a physical spherical array in a reverberant room.

Results and Discussion

Our findings highlight a critical sensitivity in DNNs to the physics of audio capture.

Frequency-Dependent Sensitivity

The research confirmed that regularization distortion is primarily a low-frequency issue. At frequencies below 1 kHz, Direction of Arrival (DOA) estimation errors were significantly higher when larger $\lambda$ values were applied. Because radial functions are smallest at these frequencies, the "cleanup" process introduces the most severe errors here, confusing models that expect clean spatial cues.

Performance and Robustness

In simulated environments, the DOA estimation error for the uninformed model nearly doubled as the regularization parameter $\lambda$ increased from 0.01 to 1. In contrast, the informed model maintained stable performance. On the real-world LOCATA dataset, the informed model achieved a consistently lower error rate (approximately 4-8 degrees), while the uninformed model's error climbed toward 12-14 degrees as more signal bins were processed.

Conclusion

This study demonstrates that the mathematical methods used to regularize Ambisonic signals have a profound impact on the reliability of downstream AI models. By providing networks with information regarding the distortion levels—essentially shielding the model from "garbage" data—we can create much more robust spatial audio systems. Future work will explore whether this informed approach can be generalized to other tasks, such as source separation and speech enhancement, to establish a more physics-aware framework for spatial audio AI.

Summary updated

Abstract

Ambisonics, a popular format of spatial audio, is the spherical harmonic (SH) representation of the plane wave density function of a sound field. Many algorithms operate in the SH domain and utilize the Ambisonics as their input signal. The process of encoding Ambisonics from a spherical microphone array involves dividing by the radial functions, which may amplify noise at low frequencies. This can be overcome by regularization, with the downside of introducing errors to the Ambisonics encoding. This paper aims to investigate the impact of different ways of regularization on Deep Neural Network (DNN) training and performance. Ideally, these networks should be robust to the way of regularization. Simulated data of a single speaker in a room and experimental data from the LOCATA challenge were used to evaluate this robustness on an example algorithm of speaker localization based on the direct-path dominance (DPD) test. Results show that performance may be sensitive to the way of regularization, and an informed approach is proposed and investigated, highlighting the importance of regularization information.
BibTeX
@inproceedings{shaybet2024ambisonics,
  author = {Bar Shaybet and Anurag Kumar and Vladimir Tourbabin and Boaz Rafaely},
  title = {{Ambisonics Networks - The Effect Of Radial Functions Regularization}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=EV7c948B6E},
}
Figure