A comparison of normalization and training approaches for ASR-dependent speaker identification

Alex Park, Timothy J. HazenInterspeech 2004

TL;DR

This paper introduces "ASR-dependent speaker identification," a method that leverages phonetic information from speech recognition to improve speaker identification accuracy. They propose a new "phone adaptive normalization" technique that outperforms previous methods, especially with limited training data, and demonstrate that training speaker models with automatically generated transcriptions yields results comparable to manual transcriptions, enabling unsupervised training.

What Problem Does This Paper Address?

Traditional text-independent speaker identification methods often treat speech as a generic signal, ignoring its phonetic content. This can limit their effectiveness, especially when dealing with variations in speech and limited training data for individual speakers. The authors tackle this by proposing "ASR-dependent speaker identification," which integrates phonetic information derived from Automatic Speech Recognition (ASR) systems into speaker models. This approach aims to build more robust and accurate speaker identification systems by understanding what is being said, not just who is saying it. A key challenge within this framework is handling situations where a speaker has very little enrollment data for specific phonetic sounds, leading to poorly trained acoustic models for those sounds. The paper also addresses the practical issue of whether to use manually transcribed audio or automatically generated transcriptions for training these speaker models.

Methods

The core of the paper is the "ASR-dependent speaker identification" approach. Instead of a single global model per speaker, this method uses a set of context-dependent (CD) acoustic models that mirror those used in ASR. During testing, an utterance is first transcribed by an ASR system, and then the phonetic information from this transcription is used to score the utterance against each speaker's CD models.

Two novel normalization techniques are presented to address the issue of sparse enrollment data:

  • Speaker Adaptive (SA) Normalization: This method interpolates speaker-dependent (SD) phonetic probabilities with speaker-independent (SI) probabilities. When enrollment data for a specific phonetic unit is scarce, it "backs off" to the SI model.
  • Phone Adaptive (PA) Normalization: This technique interpolates speaker-dependent phonetic probabilities with a speaker-dependent global GMM. This approach is intuitively more satisfying as it biases the score towards the global speaker model when phone-specific data is limited, effectively backing off to a general speaker model rather than a neutral score.

The paper also investigates the impact of training data:

  • Manually Trained (MT) Models: Trained using accurate, human-generated transcriptions of enrollment data.
  • Automatically Trained (AT) Models: Trained using transcriptions generated by the ASR system itself.

Datasets

The experiments were conducted using a corpus of speaker-labeled data collected from the MIT MERCURY airline travel information system and the MIT ORION task delegation system.

  • Enrollment Data: 44 "known" users from these systems, with varying amounts of enrollment data (less than 100 utterances, 100-500 utterances, or over 500 utterances).
  • Test Data: Calls made to the MERCURY system over a 10-month period, including utterances from known users and "out-of-set" speakers.

Key Insights and Ideas

  • Phonetic Context Matters: Integrating phonetic information from ASR significantly enhances speaker identification, moving beyond traditional text-independent methods.
  • Phone Adaptive Normalization is Superior: The new PA normalization technique offers better performance, particularly for speaker verification tasks and in scenarios with limited enrollment data, by providing more stable scores.
  • Unsupervised Training is Viable: Training speaker models using automatically generated transcriptions yields performance comparable to using manual transcriptions. This is a crucial finding for practical, large-scale deployments, as it enables unsupervised training without requiring costly manual annotation.
  • Phonetic Error Correlation: While higher phonetic error rates in ASR are correlated with poorer speaker discriminability, using AT models doesn't worsen this correlation compared to MT models, suggesting robustness.

Key Results

  • Normalization Performance: For closed-set speaker identification, both SA and PA normalization performed equally well when ample enrollment data was available. However, PA normalization showed better performance with limited enrollment data.
  • Speaker Verification: In speaker verification, PA normalization demonstrated significantly lower false acceptance rates than SA normalization at permissive operating points (low false rejection rates), making it more suitable for conversational systems where convenience is key. For instance, at a 1% false rejection rate, PA resulted in a 24.6% false acceptance rate compared to 60.3% for SA.
  • Training Data Impact: Table 2 shows that the choice between manual and automatic transcriptions for training had minimal impact on speaker ID error rates for both SA and PA normalization. For PA normalization, AT models even slightly outperformed MT models (9.36% vs. 9.61% error rate).

Limitations and Future Work

  • Limitations: The paper acknowledges that the performance of ASR-dependent speaker identification is inherently tied to the accuracy of the ASR system. Phonetic recognition errors can still impact performance, although the proposed methods mitigate this. The study focuses on specific datasets and systems, and generalization to other domains might require further validation.
  • Future Work: The authors suggest that further exploration of score normalization techniques and the integration of more sophisticated ASR outputs could lead to even greater improvements. The potential for unsupervised training opens doors for more scalable and cost-effective speaker identification systems.

Summary updated

BibTeX
@inproceedings{park2004comparison,
  author = {Alex Park and Timothy J. Hazen},
  title = {{A comparison of normalization and training approaches for ASR-dependent speaker identification}},
  booktitle = {Interspeech},
  year = {2004},
}
Figure