Selection of TDOA parameters for MDM speaker diarization

Beatriz Martínez-González, José M. Pardo, Julián D. Echeverry-Correa, José A. Vallejo-Pinto, Roberto Barra-ChicoteInterspeech 2012

TL;DR

This paper tackles the challenge of improving speaker diarization in multi-distant microphone (MDM) settings by intelligently selecting Time Delay of Arrival (TDOA) features. The authors propose and evaluate five novel methods for selecting optimal microphone pairs to calculate TDOAs, demonstrating significant improvements in diarization error rate (DER) and computational efficiency compared to a baseline system.

What Problem Does This Paper Address?

Speaker diarization, the task of identifying "who spoke when," is crucial for transcribing meetings and other multi-speaker audio. When dealing with multiple distant microphones (MDM), a common approach is to use acoustic features like Mel-Frequency Cepstral Coefficients (MFCCs) combined with localization features such as Time Delays of Arrival (TDOAs). However, traditional MDM diarization systems often rely on a single reference microphone to calculate TDOAs, potentially missing valuable information from other microphone pairs. This paper addresses the gap by exploring methods to select a more informative subset of microphone pairs for TDOA calculation, aiming to improve both accuracy and efficiency.

Methods

The core of this work lies in developing and evaluating five distinct methods for selecting which microphone pairs to use for TDOA estimation. These methods aim to move beyond the baseline's single-reference-channel approach:

  • Baseline System: Calculates TDOAs between one selected reference channel and all other channels. The reference channel is chosen based on the highest average cross-correlation.
  • Dynamic Margin (DM): Creates histograms of TDOA values for each microphone pair, discarding bins with fewer than 25 samples. It then selects pairs with the highest "dynamic range" (difference between max and min delay).
  • Extreme Regions (ER): Similar to DM, it generates TDOA histograms. It discards the most extreme delay values (0.5%) and then defines two regions containing 40% of the remaining delays. Pairs with the most values in these "target" regions are selected.
  • Most Common (MC): Calculates TDOA values for all possible microphone pairs and identifies the most frequent delay values. Pairs that contribute most to these common delays are selected.
  • Cross Correlation (XCorr): Leverages the average cross-correlation between pairs of microphones over several blocks. Pairs with high cross-correlation are chosen, aiming to avoid unreliable delay estimations.
  • Principle Component Analysis (PCA): Initially uses the XCorr method to select a larger set of microphone pairs (50). Then, it applies PCA to reduce the dimensionality of the TDOA features, aiming to capture the most significant information.

All proposed methods, after selecting the optimal microphone pairs, calculate TDOAs for each frame (10ms) and use these along with MFCCs in a subsequent segmentation and clustering process.

Datasets

  • Development Set: A combination of 12 meetings from NIST Rich Transcription 2002-2005 (devel06) and RT06/RT07 sets, forming a total of 28 meetings named ALL0607.
  • Evaluation Set: The RT09 set from the NIST Rich Transcription Evaluation of 2009, consisting of 7 meetings.

The official NIST-defined segments were used for performance measurement.

Key Insights and Ideas

  • Beyond the Single Reference: The paper strongly suggests that relying on a single reference microphone for TDOA calculation is suboptimal. Utilizing multiple, carefully selected pairs can capture richer spatial and temporal information.
  • TDOA Robustness: Even when using only TDOA features (MFCC weight = 0), the XCorr and PCA methods showed better performance than the baseline, indicating the inherent robustness of well-selected TDOA information.
  • Trade-off Between Dimensionality and Performance: Methods that reduce the TDOA vector dimension (DM, ER, XCorr) not only improve accuracy but also significantly reduce computational cost. PCA, while achieving good accuracy, incurs higher computational cost due to its initial selection of many pairs.
  • Optimal Feature Weighting: The optimal weighting between MFCC and TDOA features shifted slightly for the improved methods, favoring a bit more TDOA contribution (e.g., 0.85 MFCC / 0.15 TDOA) compared to the baseline (0.9 MFCC / 0.1 TDOA).

Key Results

The paper presents significant improvements in Diarization Error Rate (DER) on both development and evaluation sets:

  • Development Set (ALL0607):
    • XCorr achieved a 10.67% relative improvement.
  • PCA achieved a 12.45% relative improvement.
  • DM and ER also showed improvements of 5.34% and 4.41% respectively.
  • Evaluation Set (RT09):
    • XCorr achieved a substantial 36.72% relative improvement.
  • ER achieved a 10.77% relative improvement.
  • DM achieved a 3.49% relative improvement.
  • MC showed a slight degradation (-2.15%).
  • Computational Cost:
    • XCorr reduced computational time by approximately 20% compared to the baseline.
  • DM and ER also offered computational savings.
  • PCA, however, increased computational time by 24% due to its extensive initial pair selection.

The XCorr method is highlighted as the best option, offering a strong balance between performance improvement and computational efficiency.

Limitations and Future Work

  • Computational Cost of PCA: While PCA achieves good results, its high computational cost due to the initial selection of 50 pairs is a limitation.
  • MC Method Performance: The MC method performed well on the development set but not on the test set, suggesting its sensitivity to dataset characteristics.
  • Future Work: The paper implicitly suggests further exploration of feature selection techniques and potentially combining different selection criteria to further optimize TDOA feature usage. The authors also mention that the computational cost for XCorr is 20% less than the baseline, implying that further optimization might be possible.

Summary updated

BibTeX
@inproceedings{martinezgonzalez2012selection,
  author = {Beatriz Martínez-González and José M. Pardo and Julián D. Echeverry-Correa and José A. Vallejo-Pinto and Roberto Barra-Chicote},
  title = {{Selection of TDOA parameters for MDM speaker diarization}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure