Abstract
Speaker diarization, the process of determining "who spoke when," remains a significant challenge in meeting environments characterized by low signal-to-noise ratios and reverberation. This paper proposes a novel two-step speaker clustering method tailored for Single Distant Microphone (SDM) recordings. By integrating a randomized initialization strategy with a consensus-based "purification" process, the proposed system achieves a Diarization Error Rate (DER) of 16.4% on the NIST RT-09 benchmark, significantly outperforming traditional hierarchical clustering methods.
Introduction
In meeting scenarios, audio is often captured via a Single Distant Microphone (SDM), which introduces complexities such as low Signal-to-Noise Ratios (SNR), significant reverberation, and overlapping speech. Traditional diarization approaches, such as Agglomerative Hierarchical Clustering (AHC), face several hurdles in these environments:
- Computational Complexity: Starting with numerous small segments leads to expensive initial merging.
- Stopping Criteria: Determining the optimal number of speakers without prior knowledge is difficult.
- Cluster Impurity: Initial segments containing multiple speakers can "pollute" clusters, making subsequent correction nearly impossible.
System Architecture
The proposed system is divided into a specialized front-end and a novel two-step clustering back-end.
Front-End Processing
To mitigate the adverse effects of distant microphone capture, the system employs:
- Wiener Filtering: Used to track and suppress background noise.
- Feature Extraction: Mel-frequency cepstral features (MFCCs) are extracted, followed by Voice Activity Detection (VAD) to isolate speech regions.
Two-Step Speaker Clustering
The core innovation lies in the clustering logic:
- Randomized Initialization: Instead of processing every segment, the system selects a subset of long segments (4.5–5 seconds) likely to contain a single speaker. These are merged iteratively using the Bayesian Information Criterion (BIC) until a target number of clusters is reached. This approach reduces computational load and bypasses complex stopping criteria.
- Consensus-Based Purification: Inspired by resampling techniques, this step runs clustering multiple times with varying Gaussian Mixture Model (GMM) complexities. Only segments that consistently cluster together across different runs are retained. Inconsistent segments are flagged as "impure" and discarded, resulting in sharper speaker profiles.
Experimental Results
The system was evaluated on the NIST Rich Transcription 2009 (RT-09) Meeting Recognition dataset for the SDM task. The proposed method achieved a Diarization Error Rate (DER) of 16.4% and a Speaker Error (SE) of 7.8%.
The purification step proved critical, reducing the DER from 34.4% (post-initial clustering) to 16.41%. As shown in the cluster distribution analysis, the system effectively identifies dominant speaker clusters while allowing for the pruning of insignificant "noise" clusters. While the system correctly estimated the number of speakers for most recordings, it struggled with high-density environments (e.g., estimating 7 speakers instead of 11 in one specific case), suggesting areas for future refinement.
Conclusion
The two-step clustering approach offers a robust solution for SDM speaker diarization. By prioritizing initial segment length and model stability, the system effectively handles the inherent noise of meeting environments. Future work will focus on refining the consensus method to better manage high speaker density and extremely crowded audio environments.
Summary updated