Abstract
This paper introduces GP-Steerer, a novel probabilistic framework designed to upsample audio steering vectors from sparse measurements. By integrating Gaussian Process Regression (GPR) with physics-aware "deep" composite kernels, the model embeds acoustic propagation laws directly into its covariance structure. This approach enables high-fidelity spatial audio reconstruction and uncertainty quantification, achieving performance comparable to dense measurement sets while using nearly 10x fewer data points. GP-Steerer effectively bridges the gap between purely data-driven neural fields and classical physics-based models, offering a robust solution for augmented listening devices such as AR glasses and hearing aids.
Introduction
Augmented listening systems rely on accurately modeling how sound from various directions interacts with a user's microphones. This relationship is captured by Steering Vectors (SVs). In real-world scenarios, the user's head and torso scatter sound waves, creating complex, frequency-dependent patterns that are difficult to model analytically. Traditionally, capturing these patterns requires exhaustive measurements at hundreds of spatial points in controlled lab environments—a process that is both time-consuming and expensive.
Existing solutions typically fall into two categories: classical interpolation, which struggles with spatial "blind spots," and deep learning-based neural fields, which may overfit or ignore fundamental physical constraints. GP-Steerer addresses these limitations by combining the flexibility of neural networks with the rigorous probabilistic framework of Gaussian Processes.
Methods: The GP-Steerer Framework
The core of GP-Steerer is a Gaussian Process Regression (GPR) model. Unlike standard neural networks that provide only point estimates, GPR yields a probability distribution, enabling the system to quantify its own uncertainty—essentially "knowing when it doesn't know."
The primary innovation is the Deep Composite Kernel, which decomposes the complex acoustic field into three distinct physical components:
- Spectral Kernel: Ensures the sound profile changes smoothly across frequencies.
- Directional Kernel: Encodes the basic physics of a wave traveling through free space.
- Scattering Kernel: This "deep" component uses a Neural Field (NF) to estimate Spherical Harmonic coefficients, representing the complex scattering caused by the human head.
By baking "easy" physics (free-field propagation) into the kernel and letting the neural network learn the "hard" physics (scattering), GP-Steerer becomes incredibly data-efficient. This "physics-consistent" design ensures that the model's outputs remain physically plausible even when training data is sparse.
Datasets and Key Results
The authors evaluated GP-Steerer using the SPEAR Challenge dataset, which features a 6-microphone array (4 on AR glasses, 2 in-ear). The model was tested on its ability to interpolate Acoustic Transfer Functions (ATFs) across 1,020 directions on a spherical grid.
- Extreme Data Efficiency: GP-Steerer achieved "oracle" performance—matching a system trained on the full 1,020-point dataset—using only 128 measurements (less than 10%). Even with as few as 8 measurements, it produced physically sensible results.
- Superior Interpolation: The model outperformed Nearest Neighbors, Splines, and standard Neural Fields, particularly at higher frequencies (above 2 kHz) where scattering effects are most complex.
- Enhanced Beamforming: In speech enhancement tests, the upsampled steering vectors led to more precise digital beam patterns, resulting in higher Signal-to-Distortion Ratios (SDR) and improved speech quality scores (PESQ).
Discussion and Future Work
GP-Steerer demonstrates that physics should be treated as a structural constraint rather than just a loss term. This approach allows for resolution-free upsampling, enabling the model to be queried at any spatial coordinate.
However, challenges remain. The $O(N^3)$ computational complexity of Gaussian Processes may limit scalability for massive microphone arrays. Future research could explore approximation techniques or treat microphone positions as latent variables to handle hardware misalignments. Additionally, while objective metrics are strong, the authors note that perceptual validation through human listening tests is the necessary next step for real-world augmented listening applications.
Conclusion
GP-Steerer represents a significant advancement in spatial audio processing. By marrying probabilistic statistics with deep physical priors, it provides a data-efficient, robust, and physically consistent method for steering vector estimation, paving the way for more accurate and responsive augmented listening technologies.
Summary updated