Multiple scattering ambisonics: Three-dimensional sound field estimation using interacting spheres

Shoken Kaneko, Ramani Duraiswami

TL;DR

This paper introduces "Multiple Scattering Ambisonics" (MS-HOA), a novel method for capturing 3D sound fields using multiple interacting spherical microphone arrays. By accounting for the acoustic interactions between arrays, MS-HOA significantly expands the "sweet-spot" – the region where sound is accurately reconstructed – compared to traditional methods, paving the way for more immersive audio experiences in VR/AR.

What Problem Does This Paper Address?

Immersive technologies like virtual and augmented reality rely heavily on realistic spatial audio to enhance user experience. Ambisonics is a popular framework for capturing and reproducing 3D sound fields, often using spherical microphone arrays (RSMAs). However, a key limitation of traditional ambisonics is the shrinking "sweet-spot" at higher frequencies, meaning the accuracy of the reconstructed sound field degrades rapidly as the listener moves away from the ideal listening position. To overcome this, researchers want to combine data from multiple RSMAs to create a larger sweet-spot. The challenge is that when RSMAs are placed close together, they acoustically interact (scatter sound off each other), which complicates the process of combining their captured data. This paper addresses this challenge by proposing a method that explicitly models these inter-array acoustic interactions.

Methods

The core of the proposed method, MS-HOA, lies in its sophisticated handling of acoustic interactions. It builds upon the concept of "multiple scattering," where sound waves bounce off multiple objects (in this case, the microphone arrays). The paper formulates this as a "forward problem" (predicting the sound field given an incident source) and an "inverse problem" (estimating the incident sound field from microphone measurements).

Key techniques include:

  • Spherical Harmonics: The sound field is represented using spherical harmonics, a standard basis for describing functions on a sphere.
  • Translation Operators: To account for the relative positions of multiple arrays, translation operators are used to move the sound field representation from one array's coordinate system to another.
  • System Matrix: A large "system matrix" is constructed that encapsulates the interactions between all the spherical scatterers (the RSMAs). This matrix includes "single scattering matrices" for each individual RSMA and "inter-sphere translation operators" that describe how sound scatters between different RSMAs.
  • Regularized Least Squares: The inverse problem of estimating the sound field coefficients from microphone measurements is solved using regularized least squares, a common technique to handle ill-posed problems and prevent overfitting.

What's new is the explicit modeling of multiple scattering between the arrays, moving beyond simpler models that assume arrays are acoustically independent or only consider single scattering.

Datasets

Not discussed in the paper. The experiments are based on simulated sound fields generated by a monopole source.

Key Insights and Ideas

  • Inter-array interaction is crucial: The paper strongly demonstrates that ignoring acoustic interactions between closely spaced microphone arrays leads to significant degradation in sound field reconstruction accuracy, especially for expanding the sweet-spot.
  • Multiple scattering is the key: By explicitly modeling how sound scatters off multiple arrays, the MS-HOA method can achieve a much larger and more accurate sweet-spot.
  • Leveraging existing hardware: The method is designed to work with existing Rigid Spherical Microphone Arrays (RSMAs), allowing for upgrades to current systems rather than requiring entirely new hardware.
  • Higher degrees-of-freedom audio: The approach opens possibilities for more advanced spatial audio experiences that can better handle listener movement and provide a more consistent immersive experience.

Key Results

The paper presents numerical experiments demonstrating the effectiveness of MS-HOA.

  • Sweet-spot Expansion: Figures 3 and 4 show that MS-HOA significantly expands the sweet-spot area (SSA) compared to conventional HOA and single-scattering methods. For instance, in Figure 3, the SSA for MS-HOA is 0.606 m², compared to 0.019 m² for single scattering and 0.080 m² for HOA.
  • Improved SDR: The Signal-to-Distortion Ratio (SDR) maps in Figures 3 and 4 show that MS-HOA achieves higher SDR values over a larger area, indicating a more accurate reconstruction of the incident sound field.
  • Linear and Planar Grids: The method shows benefits for both linear arrangements of RSMAs (like a line of speakers) and planar grids, suggesting versatility.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, the complexity of modeling multiple scattering suggests that computational cost could be a factor, especially with a large number of arrays.
  • Hardware Cost: The paper mentions that developing RSMAs with a larger number of microphones is an effective but costly approach. MS-HOA offers an alternative by combining existing RSMAs.
  • Future Research: The authors suggest that future work should focus on reducing the cost of MS-HOA recording in terms of hardware, computation, and bandwidth for practical applications. They also highlight the importance of developing techniques for higher DoF MR/VR where listener translation needs to be supported.

Summary updated

Abstract

Rigid spherical microphone arrays (RSMAs) have been widely used in ambisonics [Gerzon (1973). J. Audio Eng. Soc. 21, 2-10] sound field recording. While it is desired to combine the information captured by a grid of densely arranged RSMAs for expanding the area of accurate reconstruction, or sweet-spots, this is not trivial due to inter-array interference. Here, we propose multiple scattering ambisonics, a method for three-dimensional ambisonics sound field recording using multiple acoustically interacting RSMAs. Numerical experiments demonstrate the sweet-spot expansion realized by the proposed method. The proposed method can be used with existing RSMAs as building blocks and opens possibilities including higher degrees-of-freedom spatial audio.
BibTeX
@inproceedings{kaneko2021multiple,
  author = {Shoken Kaneko and Ramani Duraiswami},
  title = {{Multiple scattering ambisonics: Three-dimensional sound field estimation using interacting spheres}},
  booktitle = {JASA},
  year = {2021},
  doi = {10.1121/10.0005832},
}
Figure