The LOCATA Challenge: Acoustic Source Localization and Tracking

Christine Evers, Heinrich W. Löllmann, Heinrich Mellmann, Alexander Schmidt, Hendrik Barfuss, Patrick A. Naylor, Walter Kellermann

Abstract

The ability to localize and track acoustic events is a fundamental prerequisite for equipping machines with the ability to be aware of and engage with humans in their surrounding environment. However, in realistic scenarios, audio signals are adversely affected by reverberation, noise, interference, and periods of speech inactivity. In dynamic scenarios, where the sources and microphone platforms may be moving, the signals are additionally affected by variations in the source-sensor geometries. In practice, approaches to sound source localization and tracking are often impeded by missing estimates of active sources, estimation errors, as well as false estimates. The aim of the LOCAlization and TrAcking (LOCATA) Challenge is an open-access framework for the objective evaluation and benchmarking of broad classes of algorithms for sound source localization and tracking. This article provides a review of relevant localization and tracking algorithms and, within the context of the existing literature, a detailed evaluation and dissemination of the LOCATA submissions. The evaluation highlights achievements in the field, open challenges, and identifies potential future directions.

Introduction: The Challenge of Real-World Acoustics

In fields ranging from robotics to smart home technology and hearing aids, the ability of a machine to identify "who is speaking and where" is critical. While numerous algorithms for Acoustic Source Localization and Tracking (ASLT) exist, they have historically been evaluated primarily on clean, simulated data. Real-world environments present significant hurdles: reverberation from walls, background noise, and intermittent speech patterns. Furthermore, dynamic scenarios where both the speaker and the microphone array are in motion introduce complex geometric variations.

The LOCATA (LOCAlization And TrAcking) Challenge was established to bridge this gap by providing a standardized, open-access framework and a real-world dataset annotated with high-precision ground truth. While current algorithms excel at tracking static sources, achieving approximately 1° accuracy, the challenge reveals that moving sources and sensors remain a significant hurdle for state-of-the-art acoustic signal processing.

Methodology and Algorithmic Approaches

The challenge evaluated 16 submissions employing a wide array of signal processing and machine learning techniques:

  • Localization Techniques: Core methods included Time Delay Estimation (TDE), MUltiple SIgnal Classification (MUSIC)—a subspace-based approach—and Steered Response Power with Phase Transform (SRP-PHAT), known for its robustness in reverberant conditions.
  • Spatial Domain Processing: For spherical arrays, several teams utilized spherical harmonics to capture 3D spatial information more effectively.
  • Tracking Frameworks: To manage temporal dynamics, teams implemented Kalman Filters and Particle Filters. Some advanced submissions used Probability Hypothesis Density (PHD) filters to track a varying number of concurrent speakers.
  • Machine Learning: Emerging approaches explored Deep Neural Networks (DNNs) to map raw or processed audio features directly to source coordinates.

The LOCATA Corpus and Tasks

The LOCATA corpus is built upon recordings from four distinct microphone configurations, providing a diverse set of acoustic perspectives:

  1. Robot Head: 12 microphones embedded in a NAO humanoid head.
  2. Eigenmike: A 32-microphone spherical array.
  3. DICIT Array: A 15-microphone planar array designed for distant-talking interfaces.
  4. Hearing Aids: A binaural setup with two microphones on each side of a head-torso simulator.
Overview of the microphone arrays used in the LOCATA challenge, including the robot head, DICIT array, and hearing aid simulator.

The challenge is structured into six tasks of increasing complexity, ranging from tracking a single static loudspeaker to multiple moving human talkers recorded by a moving robot platform. Ground truth positions were established using an OptiTrack motion capture system, ensuring millimeter-level accuracy.

The six tasks of the LOCATA challenge, categorized by the number of sources and the mobility of sources and arrays.

Key Insights and Performance Drivers

The evaluation revealed several critical factors influencing ASLT performance:

  • The VAD Bottleneck: Voice Activity Detection (VAD) remains a primary challenge. Errors in VAD lead to "ghost" tracks or significant latency in identifying active speakers.
  • Geometric Advantages: Spherical arrays benefit from spherical harmonic domain processing, which offers a more natural representation of 3D space than traditional time-domain methods.
  • Human vs. Loudspeaker: There is a notable performance gap between tracking static loudspeakers and human speakers. Humans are directional sources; head movements can weaken the direct sound path, complicating tracking.
  • Temporal Memory: Algorithms incorporating tracking filters (Kalman/Particle) significantly outperformed instantaneous localization methods, particularly during speech pauses.

Evaluation Results

  • Static Scenarios: In Task 1 (single static source), the top-performing methods achieved an average azimuth error of approximately 1.0° using the robot head or DICIT array.
  • Impact of Motion: Introducing source motion (Task 3) increased errors to between 3.1° and 5.5°.
  • Moving Sensors: Task 5, involving moving arrays, proved the most difficult, with average errors rising to 9.7°.
  • Algorithm Robustness: While subspace methods like MUSIC excelled in static conditions, beamforming methods like SRP-PHAT demonstrated superior robustness against heavy reverberation.

Limitations and Future Directions

The LOCATA challenge highlights several areas for future research. All data was recorded in a single laboratory environment; future benchmarks should include varied acoustic signatures to test generalization. Additionally, most tasks focused on far-field sources; future work must address "near-field" scenarios where wave curvature becomes significant. Finally, developing VAD algorithms that are natively integrated into the tracking framework could mitigate current bottlenecks, while larger real-world datasets are needed to fully realize the potential of deep learning in acoustic tracking.

Summary updated

Abstract

The ability to localize and track acoustic events is a fundamental prerequisite for equipping machines with the ability to be aware of and engage with humans in their surrounding environment. However, in realistic scenarios, audio signals are adversely affected by reverberation, noise, interference, and periods of speech inactivity. In dynamic scenarios, where the sources and microphone platforms may be moving, the signals are additionally affected by variations in the source-sensor geometries. In practice, approaches to sound source localization and tracking are often impeded by missing estimates of active sources, estimation errors, as well as false estimates. The aim of the LOCAlization and TrAcking (LOCATA) Challenge is an open-access framework for the objective evaluation and benchmarking of broad classes of algorithms for sound source localization and tracking. This article provides a review of relevant localization and tracking algorithms and, within the context of the existing literature, a detailed evaluation and dissemination of the LOCATA submissions. The evaluation highlights achievements in the field, open challenges, and identifies potential future directions.
BibTeX
@article{evers2020locata,
  author = {Christine Evers and Heinrich W. Löllmann and Heinrich Mellmann and Alexander Schmidt and Hendrik Barfuss and Patrick A. Naylor and Walter Kellermann},
  title = {{The LOCATA Challenge: Acoustic Source Localization and Tracking}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2020},
  url = {https://openreview.net/forum?id=GVpOup6I-S4},
}
Figure