Signature cluster model selection for incremental Gaussian mixture cluster modeling in agglomerative hierarchical speaker clustering

Kyu Jeong Han, Shrikanth S. NarayananInterspeech 2009 · DOI

Introduction

Agglomerative Hierarchical Speaker Clustering (AHSC) is a standard technique for grouping audio segments by speaker. It operates by recursively merging the most similar clusters. However, AHSC suffers from a significant "contamination" problem: a single incorrect merge early in the process introduces noise into the cluster's statistical model. Because there is no mechanism to undo a merge, this error propagates, leading to a cascade of further misclassifications. This study proposes "signature cluster model selection" to mitigate this issue by purifying cluster models during the merging process.

Methodology: Signature Cluster Model Selection

The proposed approach builds on incremental Gaussian Mixture Models (iGMMs). As clusters are merged, the number of Gaussian components representing the speaker's voice grows. To prevent the inclusion of "noisy" components from incorrect merges, the authors introduce a selection step that identifies the $N$-best "signature" components to represent the cluster in subsequent iterations.

Two primary strategies for signature selection are explored:

  1. Global Likelihood Comparison (GLC): This method selects components that best represent the entire dataset within the cluster.
  2. Local Likelihood Comparison (LLC): This method selects components based on their membership level to specific original data segments, providing a more localized filter.
Figure 5: Comparison of Global Likelihood Comparison (GLC) and Local Likelihood Comparison (LLC) strategies for signature model selection.

By discarding outlier components, the model maintains a "pure" representation of the speaker, making the system more robust to early-stage errors.

Experimental Evaluation

The methods were tested on 17 speech datasets from the ICSI, NIST, and ISL corpora, totaling approximately 4 hours of meeting audio. The evaluation focused on the "Speaker Error Time Rate" to measure clustering accuracy.

The results demonstrate that signature selection significantly outperforms standard AHSC. GLC achieved its best performance (10.27% error) with $N=16$, while LLC was most effective (11.01% error) with a smaller $N=4$. Overall, the proposed methods yielded a 20% relative improvement in clustering accuracy compared to the baseline.

Figure 6: Speaker Error Time Rate across 17 data sources, showing the performance of AHSC with and without signature cluster model selection (SCMS).

Conclusion and Future Work

Signature cluster model selection effectively addresses the contamination problem in AHSC. While GLC offers the highest raw accuracy, LLC is more stable across different data types and requires fewer components to be effective. A current limitation is the need to manually set the parameter $N$. Future research will focus on adaptive model selection, where the system automatically determines the optimal number of signature components based on the available data and speaker characteristics at each recursion step.

Summary updated

BibTeX
@inproceedings{han2009signature,
  author = {Kyu Jeong Han and Shrikanth S. Narayanan},
  title = {{Signature cluster model selection for incremental Gaussian mixture cluster modeling in agglomerative hierarchical speaker clustering}},
  booktitle = {Interspeech},
  year = {2009},
  doi = {10.21437/interspeech.2009-671},
}
Figure