TDOA estimation for multiple speakers in underdetermined case

Mariem Bouafif, Zied LachiriInterspeech 2012

TL;DR

This paper introduces a novel method for estimating the time delay of arrival (TDOA) for multiple speakers in challenging "underdetermined" scenarios, where the number of microphones is less than the number of speakers. By analyzing the excitation characteristics of speech using Hilbert envelopes of linear prediction residuals, their approach effectively identifies both the number of speakers and their precise arrival times, outperforming traditional angular spectrum methods.

What Problem Does This Paper Address?

Accurately pinpointing the location of sound sources, particularly in complex acoustic environments, is crucial for many applications like speech enhancement, speaker diarization, and audio forensics. A significant challenge arises in "underdetermined" scenarios, where there are more sound sources (e.g., multiple speakers) than available microphones. In such cases, traditional methods struggle to disentangle overlapping signals and reliably estimate the time delay of arrival (TDOA) for each individual source. Existing approaches often require prior knowledge of the number of speakers or are susceptible to errors introduced by reverberation, which creates spurious peaks in signal analysis. This paper aims to address these limitations by developing a robust TDOA estimation method that can handle underdetermined mixtures without needing to know the number of speakers beforehand and is more resilient to reverberation.

Methods

The core of the proposed method lies in analyzing the "excitation component" of speech signals, which is considered more robust than the raw speech signal itself for TDOA estimation. Here's a breakdown of their approach:

  1. Linear Prediction (LP) Residuals: They first compute the LP residuals for the mixed speech signals captured by two microphones. LP analysis models speech as a linear combination of past samples, and the residual represents the error between the actual speech and its prediction. This residual captures the fundamental excitation of the vocal cords.
  2. Hilbert Envelope (HE): The Hilbert envelope of these LP residuals is then calculated. The HE essentially captures the amplitude modulation of the signal, highlighting periods of significant vocal activity.
  3. Preprocessing for Excitation Emphasis: To further enhance the distinct excitation instants and mitigate the impact of reverberation, the HEs are preprocessed. This involves squaring the HE values to reduce dynamic range and then dividing by a running mean to suppress spurious peaks. This step aims to isolate the most prominent excitation events.
  4. Cross-Correlation of Modified HEs: Finally, the time delays of arrival are estimated by computing the cross-correlation between the preprocessed Hilbert envelopes from the two microphone signals. Peaks in this cross-correlation function correspond to the TDOAs of the speakers. The method inherently estimates the number of speakers by identifying the significant peaks in this correlation.

This approach is contrasted with traditional angular spectrum methods, which often rely on phase differences across frequency bins and require a predefined number of sources.

Datasets

  • TIMIT Database: Used for initial experiments and comparisons, particularly for evaluating performance in controlled multi-speaker scenarios.
  • SISEC 2010 Database: Utilized for more extensive evaluation, including tests with varying numbers of speakers, reverberation times, and microphone spacings. This database is noted for its underdetermined mixtures.

Key Insights and Ideas

  • Excitation Characteristics are Key: Focusing on the excitation component of speech (via LP residuals and their Hilbert envelopes) proves more effective for TDOA estimation than using the raw speech signal, especially in reverberant conditions.
  • Preprocessing is Crucial for Robustness: The specific preprocessing steps (squaring and running mean normalization of HEs) are vital for emphasizing genuine excitation peaks and suppressing spurious ones caused by reverberation.
  • Underdetermined Case Solved: The method successfully estimates both the number of speakers and their TDOAs without requiring prior knowledge of the speaker count, a significant advantage over many existing techniques.
  • Reverberation Remains a Challenge: While more robust than some methods, reverberation still impacts performance by introducing additional sidelobes that can complicate peak identification, leading to a decrease in F-measure at higher reverberation levels.

Key Results

  • Superior Performance: The proposed method demonstrated better performance compared to several angular spectrum methods (GCC-PHAT, MUSIC, MVDR, DS) on the TIMIT database, providing TDOA estimates closer to the actual values.
  • Accurate Speaker Counting: The method correctly identified the number of speakers and their corresponding TDOAs, whereas angular spectrum methods often require the number of speakers to be pre-specified.
  • F-Measure Performance: On the SISEC 2010 database, the F-measure (a metric combining precision and recall) was evaluated. The proposed method performed well for reverberation times up to 300ms. For higher reverberation, the F-measure decreased.
  • Microphone Spacing Impact: The algorithm showed better performance with moderate to large microphone spacings (30cm to 1m).
  • Peak Selection Importance: For angular spectrum methods, selecting a number of peaks equal to the actual number of speakers was necessary to achieve optimal F-measure, highlighting the advantage of the proposed method's inherent speaker counting capability.

Limitations and Future Work

  • Reverberation Sensitivity: The paper acknowledges that reverberation is the primary limitation, affecting the number of identifiable peaks and reducing the F-measure at higher levels.
  • Future Work Suggestion: The authors suggest that applying dereverberation techniques to the mixed speech signal before TDOA computation could further improve the method's performance.

Summary updated

BibTeX
@inproceedings{bouafif2012tdoa,
  author = {Mariem Bouafif and Zied Lachiri},
  title = {{TDOA estimation for multiple speakers in underdetermined case}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure