Abstract
This paper presents a scalable approach to speaker clustering by framing it as a community detection problem on a graph. High-dimensional "i-vector" speech representations are mapped into a $k$-nearest neighbor network, followed by a novel manifold-learning-based optimization. By incorporating a local invariance constraint within a Non-negative Matrix Factorization (NMF) framework, the proposed method captures the intrinsic non-linear geometry of speech data. Experimental results on the CH50 and NIST SRE 2008 datasets demonstrate significant improvements in accuracy and scalability over traditional Euclidean-based clustering and standard community detection algorithms.
Introduction
Speaker diarization, the task of determining "who spoke when" in an audio stream, relies heavily on effective speaker clustering. The current industry standard utilizes high-dimensional i-vectors to represent speech segments. However, traditional clustering algorithms like K-means or Agglomerative Hierarchical Clustering (AHC) operate in Euclidean space, often failing to account for the complex, non-linear manifold structures inherent in speech data. This limitation leads to degraded performance in large-scale or noisy environments.
Methodology
The proposed framework transitions from simple vector distance metrics to a network-based topology through a two-step process:
- Graph Construction: Utterances are represented as nodes in a Speaker Content Network. Edges are established between each node and its $k$-nearest neighbors ($k$-NN) based on cosine similarity, effectively capturing the local manifold structure.
- Community Detection with Local Invariance: Speakers are identified as communities within the network. The core innovation is an objective function based on Non-negative Matrix Factorization (NMF) that includes a local invariance constraint via a graph Laplacian. This constraint ensures that segments similar in the original feature space have a high probability of belonging to the same community.
- Optimization: An efficient multiplicative update rule is derived to solve the optimization problem, ensuring scalability for large datasets.
Experimental Setup
The method was evaluated on two datasets:
- CH50: A Chinese speech database with 50 speakers and 500 utterances, used for sensitivity analysis regarding network complexity.
- NIST SRE 2008: A large-scale benchmark containing 7,607 utterances from 1,230 speakers, testing robustness and scalability in a standard recognition environment.
Results and Analysis
The proposed method consistently outperformed eight baseline algorithms, including classic community detection (e.g., Infomap) and spectral clustering.
- Accuracy: On the CH50 dataset, the method achieved a 16.52% improvement in Normalized Mutual Information (NMI) over the best baseline. On the NIST SRE08 dataset, it reached an NMI of 84.16%.
- Parameter Sensitivity: The choice of $k$ is critical. Optimal performance is generally observed when $k$ is approximately half the average number of utterances per speaker, ensuring the graph is connected enough for intra-speaker segments but sparse enough to separate different speakers.
- Scalability: The algorithm processed the SRE08 dataset in roughly 40 minutes, indicating its viability for real-world applications.
Conclusion
By leveraging manifold learning and community detection, this research provides a robust alternative to Euclidean clustering for speaker diarization. The use of local invariance constraints allows the model to respect the intrinsic geometry of i-vector space. Future work will focus on adapting the model for dynamic or streaming data environments where speech segments arrive incrementally, requiring the manifold to be updated without full re-calculation.
Summary updated