Using phoneme recognition and text-dependent speaker verification to improve speaker segmentation for Chinese speech

Gang Wang, Xiaojun Wu, Thomas Fang ZhengInterspeech 2010

TL;DR

This paper proposes a novel approach to speaker segmentation by leveraging phoneme recognition to convert the problem into a text-dependent speaker verification task. This method significantly improves performance on short utterances, outperforming traditional methods like BIC and GLR.

What Problem Does This Paper Address?

Speaker segmentation, the process of dividing an audio stream into segments belonging to different speakers, is crucial for tasks like multi-speaker detection and speaker tracking. A major challenge in this field is achieving high accuracy, especially with short audio segments. Traditional methods often struggle because the limited data in short segments leads to biased performance. The authors argue that improving speaker verification (SV) for short utterances is key to improving speaker segmentation, and they propose a method to achieve this by making the SV task text-dependent.

Methods

The core of the proposed method lies in transforming the text-independent speaker segmentation problem into a text-dependent speaker verification problem. Here's how it works:

  1. Phoneme Recognition: First, an existing phoneme recognizer is used to convert the test utterance into a sequence of phonemes.
  2. Text-Dependent SV: Instead of directly comparing acoustic features of adjacent windows, the method performs text-dependent speaker verification on the phoneme sequences extracted from these windows. This is achieved using Dynamic Time Warping (DTW) to align and compare phonemes.
  3. Distance Metric: The DTW-based results are then used as a distance measure between adjacent audio windows. If the distance is large, it indicates a potential speaker change.
  4. Phoneme Weighting and Normalization: To further enhance performance, the authors introduce a weighting scheme for phonemes based on their discriminative power (using F-Ratio) and normalize the distances to ensure consistency across different phonemes.

This approach is a hybrid method, combining aspects of metric-based and model-based techniques by using phoneme recognition to bridge the gap.

Datasets

  • Chinese Corpus Consortium (CCC) MSS database: Used for multi-speaker segmentation experiments. This database consists of conversations between two speakers.
  • CCC Speaking Style database: Used for training the UBM (Universal Background Model) for speaker verification.
  • NIST Rich Transcription Evaluations: Mentioned as a context for speaker diarization.

Key Insights and Ideas

  • Bridging Text-Independent to Text-Dependent SV: The fundamental insight is that by recognizing phonemes, a text-independent segmentation problem can be effectively transformed into a text-dependent SV problem. This leverages the inherent consistency in content that text-dependent SV benefits from.
  • Short Utterance Performance: The method shows a significant advantage for short utterances, where traditional methods often falter due to insufficient data.
  • Phoneme Discriminability Matters: Recognizing that not all phonemes are equally informative for speaker verification, the authors incorporate a weighting mechanism based on phoneme discriminability.

Key Results

  • Improved Speaker Verification for Short Utterances: The proposed method, when compared to conventional GMM-UBM, achieved better performance (lower EER and MinDCF) on short utterances (3, 6, and 9 seconds). For example, for 3-second utterances, the EER dropped from 27.4% to 23.5%.
  • Enhanced Speaker Segmentation: The phoneme recognition-based method (PR) outperformed baseline methods like BIC and GLR in speaker segmentation tasks. While the False Alarm Rate (FAR) was slightly worse than BIC or GLR in some cases, the Miss Detection Rate (MDR) was significantly reduced. For the CCC MSS dataset, the PR method achieved an MDR of 21.2%, compared to 27.1% for BIC and 28.3% for GLR.
  • Effectiveness of Phoneme Weighting: Using F-Ratio based phoneme weighting improved SV performance compared to equal weighting, especially for shorter utterances.

Limitations and Future Work

  • Phoneme Set and Discriminability: The authors suggest that further research into the phoneme set and more sophisticated distance measures for phonemes could further improve the speech recognition phase, which in turn would benefit the segmentation.
  • FAR Reduction: While MDR was improved, the FAR was slightly worse than some baseline methods. The authors suggest that follow-up processing, such as clustering, could be used to reduce the FAR.
  • Computational Cost: Not explicitly discussed, but the addition of a phoneme recognition step likely increases computational complexity compared to purely metric-based methods.

Summary updated

BibTeX
@inproceedings{wang2010using,
  author = {Gang Wang and Xiaojun Wu and Thomas Fang Zheng},
  title = {{Using phoneme recognition and text-dependent speaker verification to improve speaker segmentation for Chinese speech}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure