Adaptive decision tree-based phone cluster models for speaker clustering

Chia-Hsin Hsieh, Chung-Hsien Wu, Han-Ping ShenInterspeech 2008

TL;DR

This paper introduces an adaptive decision tree-based phone cluster model (DT-PCM) for speaker clustering in broadcast news. By leveraging decision trees to share contextual and speaker information across universal phone models, their approach significantly outperforms traditional GMM-based methods, leading to more accurate speaker identification in long audio streams.

What Problem Does This Paper Address?

Managing and retrieving information from the ever-increasing volume of spoken content, like broadcast news or meeting recordings, is a significant challenge. Speaker clustering, a technique that groups segments of audio belonging to the same speaker, is crucial for annotating this content and enabling efficient retrieval. However, a key hurdle is accurately clustering speakers when speech segments are short, making it difficult to train robust speaker-specific models. Existing methods often struggle with data sparsity and the generality of universal models. This paper aims to bridge this gap by developing a more effective way to adapt universal phone models to individual speakers, even with limited data.

Methods

The core innovation lies in the Adaptive Decision Tree-based Phone Cluster Models (DT-PCMs). Here's a breakdown of their approach:

  • Universal Phone Modeling: They begin by training a set of universal phone models using a large broadcast news database. The Multi-Space Probability Distributed-Hidden Markov Model (MSD-HMM) is employed to handle both cepstral and pitch (FO) features for voiced and unvoiced phones.
  • Phone Clustering: To reduce model complexity and address confusion between similar-sounding phones, they merge confusing phone models into "phone clusters." This is done using an agglomerative clustering algorithm based on a sub-syllable similarity measure.
  • Decision Tree Construction: This is the novel part. A decision tree is built where leaf nodes store Gaussian models. These models capture contextual, phonetic, and speaker-specific characteristics. The tree allows for data sharing across universal speakers, making adaptation more efficient. The tree is trained using features like phone context, phone confusion candidates, prosodic features (duration ratios), and cepstral/FO features.
  • Speaker Adaptation: For speaker clustering, input speech segments are first recognized to obtain phone sequences. The DT-PCMs are then used to retrieve relevant Gaussian models for each state of the phone cluster MSD-HMMs. These retrieved models are then adapted to create speaker-specific phone cluster models using Maximum Likelihood Linear Regression (MLLR).
  • Speaker Clustering: Finally, the adapted speaker-related phone cluster models are used for clustering via a cross-likelihood ratio (CLR) measure. A bottom-up agglomerative clustering algorithm is employed.

Datasets

  • Training Data: MATBN (Mandarin Across Taiwan-Broadcast News) corpus. Specifically, speech segments from anchors and field reporters collected during 2001 and 2002 were used for training universal phone models. Twenty-five speakers were involved in this training.
  • Tuning Data: One hour of broadcast news from the show "20030306" was used for parameter tuning. This data included 14 speakers and 77 speech segments.
  • Test Data: One hour of broadcast news from the show "20030129" was used for testing. This data comprised 11 speakers and 74 speech segments. The average length of each speech segment was approximately 20 seconds.

Key Insights and Ideas

  • Leveraging Context and Speaker Characteristics: The DT-PCM approach effectively captures and shares crucial contextual and speaker-specific information within a unified framework, which is vital for adapting general models to individual speakers.
  • Decision Trees for Data Sharing: Using decision trees as a mechanism to store and retrieve adapted Gaussian models is a clever way to combat data sparsity and improve the efficiency of model adaptation, especially for short speech segments.
  • Phone Clustering Reduces Complexity: Merging confusing phone models into clusters simplifies the overall model structure, making it more robust and less prone to errors, particularly when dealing with acoustic variability.
  • MLLR for Efficient Adaptation: MLLR is a standard yet effective technique for adapting acoustic models, and its application here allows for quick and efficient customization of the universal models to speaker-specific characteristics.

Key Results

  • Outperforms GMM: The proposed DT-PCMs approach consistently outperformed the conventional GMM-based approach in speaker clustering.
  • Reduced Under-clustering: While GMM-based methods suffered from under-clustering (grouping too few speakers), the DT-PCMs approach showed a "slighter under-clustering problem."
  • Higher Purity: The DT-PCMs approach achieved higher average cluster purity (acp) compared to GMM.
  • Quantitative Metrics:
    • On the test set (20030129), DT-PCMs achieved an average cluster purity (acp) of 0.916 and an average speaker purity (asp) of 0.812, resulting in a geometric mean (K) of 0.865. The number of clusters (C#) was 14.
  • In contrast, GMM achieved an acp of 0.968, an asp of 0.744, and a K of 0.849, with 19 clusters. (Note: The paper states GMM had under-clustering, which is reflected in the lower asp despite a higher acp, and a higher cluster count than the true speaker count).

Limitations and Future Work

  • Under-clustering: While improved, the DT-PCMs approach still exhibits a slight under-clustering problem, suggesting room for further refinement in cluster merging or splitting strategies.
  • Threshold Tuning: The agglomerative clustering algorithm requires a threshold-based stopping criterion, necessitating a tuning set to determine the optimal threshold for testing. This adds a dependency on having a representative tuning dataset.
  • Future Work: Not explicitly detailed, but the success of this adaptive, tree-based approach suggests potential for exploring more sophisticated adaptation techniques or incorporating richer linguistic/semantic features for even greater accuracy.

Summary updated

BibTeX
@inproceedings{hsieh2008adaptive,
  author = {Chia-Hsin Hsieh and Chung-Hsien Wu and Han-Ping Shen},
  title = {{Adaptive decision tree-based phone cluster models for speaker clustering}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure