Speaker diarization in meeting audio for single distant microphone

Tin Lay Nwe, Hanwu Sun, Bin Ma, Haizhou Li

Abstract

Speaker diarization, the process of determining "who spoke when," remains a significant challenge in meeting environments characterized by low signal-to-noise ratios and reverberation. This paper proposes a novel two-step speaker clustering method tailored for Single Distant Microphone (SDM) recordings. By integrating a randomized initialization strategy with a consensus-based "purification" process, the proposed system achieves a Diarization Error Rate (DER) of 16.4% on the NIST RT-09 benchmark, significantly outperforming traditional hierarchical clustering methods.

Introduction

In meeting scenarios, audio is often captured via a Single Distant Microphone (SDM), which introduces complexities such as low Signal-to-Noise Ratios (SNR), significant reverberation, and overlapping speech. Traditional diarization approaches, such as Agglomerative Hierarchical Clustering (AHC), face several hurdles in these environments:

  1. Computational Complexity: Starting with numerous small segments leads to expensive initial merging.
  2. Stopping Criteria: Determining the optimal number of speakers without prior knowledge is difficult.
  3. Cluster Impurity: Initial segments containing multiple speakers can "pollute" clusters, making subsequent correction nearly impossible.

System Architecture

The proposed system is divided into a specialized front-end and a novel two-step clustering back-end.

System architecture overview showing front-end processing and speaker clustering stages.

Front-End Processing

To mitigate the adverse effects of distant microphone capture, the system employs:

  • Wiener Filtering: Used to track and suppress background noise.
  • Feature Extraction: Mel-frequency cepstral features (MFCCs) are extracted, followed by Voice Activity Detection (VAD) to isolate speech regions.

Two-Step Speaker Clustering

The core innovation lies in the clustering logic:

  1. Randomized Initialization: Instead of processing every segment, the system selects a subset of long segments (4.5–5 seconds) likely to contain a single speaker. These are merged iteratively using the Bayesian Information Criterion (BIC) until a target number of clusters is reached. This approach reduces computational load and bypasses complex stopping criteria.
  2. Consensus-Based Purification: Inspired by resampling techniques, this step runs clustering multiple times with varying Gaussian Mixture Model (GMM) complexities. Only segments that consistently cluster together across different runs are retained. Inconsistent segments are flagged as "impure" and discarded, resulting in sharper speaker profiles.

Experimental Results

The system was evaluated on the NIST Rich Transcription 2009 (RT-09) Meeting Recognition dataset for the SDM task. The proposed method achieved a Diarization Error Rate (DER) of 16.4% and a Speaker Error (SE) of 7.8%.

Distribution of frames across target clusters, illustrating the identification of dominant speakers and the pruning of minor clusters.

The purification step proved critical, reducing the DER from 34.4% (post-initial clustering) to 16.41%. As shown in the cluster distribution analysis, the system effectively identifies dominant speaker clusters while allowing for the pruning of insignificant "noise" clusters. While the system correctly estimated the number of speakers for most recordings, it struggled with high-density environments (e.g., estimating 7 speakers instead of 11 in one specific case), suggesting areas for future refinement.

Conclusion

The two-step clustering approach offers a robust solution for SDM speaker diarization. By prioritizing initial segment length and model stability, the system effectively handles the inherent noise of meeting environments. Future work will focus on refining the consensus method to better manage high speaker density and extremely crowded audio environments.

Summary updated

BibTeX
@inproceedings{nwe2010speaker,
  author = {Tin Lay Nwe and Hanwu Sun and Bin Ma and Haizhou Li},
  title = {{Speaker diarization in meeting audio for single distant microphone}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-440},
}
Figure