TL;DR
This paper introduces MIRAGE, a novel approach to 2D sound source localization (SSL) using just two microphones. By cleverly leveraging early acoustic echoes, MIRAGE can estimate both the azimuth and elevation of a sound source, a feat impossible with two microphones in anechoic conditions. This is achieved through a hybrid learning-based and physics-based framework.
What Problem Does This Paper Address?
Sound source localization (SSL) aims to pinpoint the location of a sound source in 3D space. Traditional methods often struggle in real-world environments due to reverberation and acoustic echoes, which distort the sound signals. While many SSL techniques treat echoes as a nuisance, this paper explores the opposite: using early echoes as a valuable source of information. Specifically, the challenge is to perform 2D SSL (estimating both azimuth and elevation) with a minimal setup of only two microphones, a task that is fundamentally limited in free-field conditions.
Methods
The core of MIRAGE is a two-part approach:
- Learning-based Echo Estimation: A deep neural network (DNN) is trained on simulated data to estimate key characteristics of early echoes. The network takes interaural level difference (ILD) and interaural phase difference (IPD) features as input and outputs three values representing the time differences of arrival (TDOA) between:
- The direct sound paths to the two microphones.
- The first echoes to the two microphones.
- The direct path to one microphone and its corresponding echo.
- Physics-based Echo Aggregation: These estimated time differences are then used within a physics-based framework. The concept of "image microphones" is employed, where virtual microphones are created as reflections of the true microphones across a reflective surface. This effectively creates a virtual microphone array that includes both the real and image microphones. The estimated TDOAs are then used to generate local angular spectra, which are aggregated to estimate the 2D direction of arrival (DOA) of the sound source.
Datasets
The authors generated a synthetic dataset for training and validation. This dataset consists of approximately 90,000 audio scenes simulated using an image-method and ray-tracing approach. These simulations incorporate various room configurations, source/microphone positions, and acoustic properties, resulting in reverberation times (RT60) between 20 ms and 250 ms. For testing, they used 200 new RIRs convolved with both white noise and speech signals, with and without added external noise.
Key Insights and Ideas
- Echoes as Information: The paper demonstrates that acoustic echoes, often considered detrimental to SSL, can actually be a rich source of information, especially for estimating elevation.
- Minimalist SSL: Achieving 2D SSL with just two microphones is a significant advancement, reducing hardware requirements and complexity.
- Hybrid Approach: Combining the pattern recognition capabilities of deep learning for echo estimation with the geometric rigor of physics-based models for aggregation proves effective.
- Image Microphone Model: The use of the image-microphone model provides a convenient way to conceptualize and utilize echo information within a virtual array.
Key Results
- Azimuth Estimation: MIRAGE performs comparably to the well-established GCC-PHAT method for azimuth estimation in the considered scenario.
- Elevation Estimation: Crucially, MIRAGE successfully estimates elevation angles, an impossible task for two microphones in anechoic settings.
- Noise Robustness: While external noise degrades performance for both MIRAGE and GCC-PHAT, the MIRAGE model shows some generalization capabilities from white noise to speech sources.
- TDOA Estimation: The DNN's TDOA estimation achieved similar nRMSE to GCC-PHAT, though estimating iTDOA and TDOE was more challenging.
Limitations and Future Work
- Simulated Data: The current results are based on simulated data. Extending the framework to real-world data is a key next step.
- Echo Delay Estimation: The paper acknowledges that echo delay estimation is challenging, and further research is needed to develop more robust estimation schemes.
- Noise and Signal Variety: Future work will focus on better handling various types of noise and emitted signals.
- Larger Arrays and Higher-Order Echoes: The authors suggest exploring MIRAGE for larger microphone arrays, higher-order echoes, and applications beyond SSL.
Summary updated