Gaussian Process Regression of Steering Vectors With Physics-Aware Deep Composite Kernels for Augmented Listening

Mathieu Fontaine, Diego Di Carlo, Shoichi Koyama, Aditya Arie Nugraha, Mathieu Fontaine 0002, Yoshiaki Bando, Kazuyoshi YoshiiarXiv 2025 · DOI · arXiv

Abstract

This paper introduces GP-Steerer, a novel probabilistic framework designed to upsample audio steering vectors from sparse measurements. By integrating Gaussian Process Regression (GPR) with physics-aware "deep" composite kernels, the model embeds acoustic propagation laws directly into its covariance structure. This approach enables high-fidelity spatial audio reconstruction and uncertainty quantification, achieving performance comparable to dense measurement sets while using nearly 10x fewer data points. GP-Steerer effectively bridges the gap between purely data-driven neural fields and classical physics-based models, offering a robust solution for augmented listening devices such as AR glasses and hearing aids.

Introduction

Augmented listening systems rely on accurately modeling how sound from various directions interacts with a user's microphones. This relationship is captured by Steering Vectors (SVs). In real-world scenarios, the user's head and torso scatter sound waves, creating complex, frequency-dependent patterns that are difficult to model analytically. Traditionally, capturing these patterns requires exhaustive measurements at hundreds of spatial points in controlled lab environments—a process that is both time-consuming and expensive.

Spatial setup for steering vector estimation on a wearable microphone array.

Existing solutions typically fall into two categories: classical interpolation, which struggles with spatial "blind spots," and deep learning-based neural fields, which may overfit or ignore fundamental physical constraints. GP-Steerer addresses these limitations by combining the flexibility of neural networks with the rigorous probabilistic framework of Gaussian Processes.

Methods: The GP-Steerer Framework

The core of GP-Steerer is a Gaussian Process Regression (GPR) model. Unlike standard neural networks that provide only point estimates, GPR yields a probability distribution, enabling the system to quantify its own uncertainty—essentially "knowing when it doesn't know."

The primary innovation is the Deep Composite Kernel, which decomposes the complex acoustic field into three distinct physical components:

  1. Spectral Kernel: Ensures the sound profile changes smoothly across frequencies.
  2. Directional Kernel: Encodes the basic physics of a wave traveling through free space.
  3. Scattering Kernel: This "deep" component uses a Neural Field (NF) to estimate Spherical Harmonic coefficients, representing the complex scattering caused by the human head.
Architectural comparison of GPSteerer against data-driven and physics-informed baselines.

By baking "easy" physics (free-field propagation) into the kernel and letting the neural network learn the "hard" physics (scattering), GP-Steerer becomes incredibly data-efficient. This "physics-consistent" design ensures that the model's outputs remain physically plausible even when training data is sparse.

Datasets and Key Results

The authors evaluated GP-Steerer using the SPEAR Challenge dataset, which features a 6-microphone array (4 on AR glasses, 2 in-ear). The model was tested on its ability to interpolate Acoustic Transfer Functions (ATFs) across 1,020 directions on a spherical grid.

  • Extreme Data Efficiency: GP-Steerer achieved "oracle" performance—matching a system trained on the full 1,020-point dataset—using only 128 measurements (less than 10%). Even with as few as 8 measurements, it produced physically sensible results.
  • Superior Interpolation: The model outperformed Nearest Neighbors, Splines, and standard Neural Fields, particularly at higher frequencies (above 2 kHz) where scattering effects are most complex.
  • Enhanced Beamforming: In speech enhancement tests, the upsampled steering vectors led to more precise digital beam patterns, resulting in higher Signal-to-Distortion Ratios (SDR) and improved speech quality scores (PESQ).

Discussion and Future Work

GP-Steerer demonstrates that physics should be treated as a structural constraint rather than just a loss term. This approach allows for resolution-free upsampling, enabling the model to be queried at any spatial coordinate.

However, challenges remain. The $O(N^3)$ computational complexity of Gaussian Processes may limit scalability for massive microphone arrays. Future research could explore approximation techniques or treat microphone positions as latent variables to handle hardware misalignments. Additionally, while objective metrics are strong, the authors note that perceptual validation through human listening tests is the necessary next step for real-world augmented listening applications.

Conclusion

GP-Steerer represents a significant advancement in spatial audio processing. By marrying probabilistic statistics with deep physical priors, it provides a data-efficient, robust, and physically consistent method for steering vector estimation, paving the way for more accurate and responsive augmented listening technologies.

Summary updated

BibTeX
@misc{fontaine2025gaussian,
  author = {Mathieu Fontaine and Diego Di Carlo and Shoichi Koyama and Aditya Arie Nugraha and Mathieu Fontaine and Yoshiaki Bando and Kazuyoshi Yoshii},
  title = {{Gaussian Process Regression of Steering Vectors With Physics-Aware Deep Composite Kernels for Augmented Listening}},
  year = {2025},
  eprint = {2509.02571},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.02571},
}
Figure