Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure

Siqi Zheng, Hongbin SuoICASSP 2022 · DOI

TL;DR

This paper reframes speaker diarization as a community detection problem, leveraging the Leiden algorithm to capture the topological structure of speaker embeddings. By integrating this with a novel masked filtering technique for cleaner embeddings and an end-to-end post-processing module, the authors achieve significant improvements in diarization accuracy, particularly in challenging scenarios with multiple speakers.

What Problem Does This Paper Address?

Speaker diarization, the task of identifying "who spoke when" in an audio recording, has seen advancements with end-to-end deep learning models. However, these models can struggle with long meetings and a large number of speakers. Traditional clustering-based approaches are more robust in these scenarios but often rely on simplistic clustering methods like k-means or spectral clustering, which fail to adequately capture the complex relationships and topological structure within speaker embeddings. This paper aims to bridge this gap by treating speaker diarization as a community detection problem, thereby incorporating richer structural information to improve clustering performance.

Methods

The core of this work lies in reformulating speaker diarization as a community detection problem. The authors propose a system with several key components:

  • Leiden Community Detection: Instead of traditional clustering methods, they employ the Leiden algorithm, a state-of-the-art community detection algorithm known for its ability to find well-connected communities. This is applied to a network where speaker embeddings are nodes and similarity scores are edge weights.
  • Uniform Manifold Approximation and Projection (UMAP): To handle the high dimensionality of speaker embeddings and preserve their global and local topological structure, UMAP is used for dimension reduction. This is preferred over t-SNE due to its better preservation of structure and flexibility in reducing to any optimal dimension.
  • "Winner Takes All" Masked Filtering: To address the issue of noisy embeddings from longer segments (which are preferred for robustness), a masked filtering technique is introduced. This method identifies the dominant speaker in a segment and uses its embedding as a target reference for extracting cleaner embeddings, effectively mitigating the impact of overlapping speech.
  • End-to-End Post-Processing (EEND-CommPart): Finally, the community partitions obtained from the clustering are fed into an end-to-end post-processing network. This module refines the diarization results, handling overlapping speech and smoothing frame-level predictions, by fixing the number of speakers to the number of detected communities.

Datasets

The experiments were conducted using simulated data derived from the NIST SRE corpus. The training corpus consisted of 57,517 utterances from 5,767 speakers from NIST SRE 04-10. For evaluation, the SRE10 Evaluation set was used, with simulated meetings of 2-8 speakers and overlap ratios ranging from 5% to 30%.

Key Insights and Ideas

  • Community Detection for Diarization: The fundamental insight is that the relationships between speaker embeddings can be effectively modeled as a network, and community detection algorithms are well-suited to uncover these structures, leading to better speaker segmentation.
  • Topology Matters: Traditional clustering methods often overlook the underlying topological structure of the embedding space. By adopting community detection, the paper emphasizes the importance of this structure for accurate diarization.
  • UMAP's Advantage: UMAP's ability to preserve both local and global topological information makes it a superior choice for dimension reduction in this context compared to methods like t-SNE.
  • Masked Filtering for Cleaner Embeddings: The "winner takes all" masked filtering approach is a clever way to leverage longer segments for robust embeddings while simultaneously cleaning out interference from other speakers.
  • EEND-CommPart Synergy: Combining the global speaker counting advantage of clustering with the local refinement capabilities of end-to-end models through EEND-CommPart proves to be a powerful strategy.

Key Results

The proposed system, System No.6 (Masked Filtering + UMAP-Leiden + EEND-CommPart), demonstrated significant performance improvements:

  • Overall Performance: System No.6 achieved the optimal performance for meetings with 4 or more speakers, outperforming baselines like EEND-EDA and k-means clustering by a remarkable margin.
  • DER Reduction: The system achieved a relative DER reduction of up to 70 percent.
  • Scalability: UMAP-Leiden performed reasonably well on larger numbers of speakers, unlike k-means and spectral clustering, which saw significant performance drops.
  • Efficiency: The Leiden algorithm was found to be more efficient than Louvain in terms of running time, as shown in Table 3.

Limitations and Future Work

  • Dataset Specificity: While the system shows strong performance on NIST SRE data, its generalization to other datasets with different acoustic conditions or speaker characteristics might require further investigation.
  • Masked Filtering Cost: The masked filtering approach, while beneficial, adds computational cost. The authors note that it can be optionally turned off for more structured meetings.
  • Future Work: The authors express interest in exploring more efficient methods for the masked filtering purpose. They also highlight that clustering-based diarization still has significant room for improvement and hope their work inspires further research in this area.

Summary updated

BibTeX
@inproceedings{zheng2022reformulating,
  author = {Siqi Zheng and Hongbin Suo},
  title = {{Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2022},
  doi = {10.1109/icassp43922.2022.9747611},
}
Figure