A cohort - UBM approach to mitigate data sparseness for in-set/out-of-set speaker recognition

Vinod Prakash, John H. L. HansenInterspeech 2006

TL;DR

This paper tackles the challenge of speaker recognition with very limited enrollment data (just 5 seconds). They propose a novel "cohort-UBM" approach that leverages acoustically similar speakers to "fill in the gaps" in a speaker's model, significantly improving recognition accuracy compared to standard methods.

What Problem Does This Paper Address?

Speaker recognition systems aim to identify who is speaking. A key challenge arises when the amount of speech data available for training a speaker's unique profile (enrollment) is very short. This "data sparseness" leads to incomplete acoustic models, often called "acoustic holes," where the model might miss certain phonetic sounds. When a test utterance contains these missing sounds, the system can incorrectly reject the speaker, especially if the speaker is from an "out-of-set" group (i.e., not one of the enrolled speakers). This paper specifically addresses the problem of distinguishing between enrolled ("in-set") speakers and unknown ("out-of-set") speakers when enrollment data is extremely limited (5 seconds) and test utterances are also short (2-8 seconds).

Methods

The authors propose a "cohort-UBM" approach. The standard baseline system uses a Universal Background Model (UBM), which is a general model trained on a large dataset of diverse speakers. Speaker-specific models are then adapted from this UBM using the limited enrollment data via Maximum A Posteriori (MAP) adaptation.

The novel contribution is the use of "cohort speakers." Instead of solely relying on the general UBM, the proposed method first identifies a set of speakers from the development data who are acoustically "close" to the target in-set speaker. This closeness is determined by scoring the target speaker's enrollment data against models built from all other development speakers. The top-scoring speakers form the "cohort set." A cohort GMM is then built by pooling data from these cohort speakers. This cohort GMM, rather than the general UBM, is then used as the initial model for MAP adaptation to create the final in-set speaker model. The idea is that these acoustically similar cohort speakers can help fill the "acoustic holes" in the target speaker's model.

Datasets

The experiments were conducted using the TIMIT database.

  • Training and Testing: A set of 60 male speakers from TIMIT were used. These speakers served as both in-set and out-of-set speakers depending on the experimental setup. Three different in-set/out-of-set configurations were used: 15 in-set/45 out-of-set, 30 in-set/30 out-of-set, and 45 in-set/15 out-of-set.
  • Enrollment Data: Limited to approximately 5 seconds of speech per speaker.
  • Test Data: Ranged from 2, 4, 6, and 8 seconds of speech.
  • Development Data: The remaining 378 male speakers (each with about 30 seconds of data) were used as development data for building the UBM and cohort models.

Key Insights and Ideas

  • Data Sparseness is a Major Hurdle: The paper strongly emphasizes that very short enrollment data (5 seconds) creates significant "acoustic holes" in speaker models, leading to poor recognition performance.
  • Acoustically Similar Speakers are Key: The core idea is that speakers who sound similar to the target speaker can provide valuable phonetic information that might be missing from the target speaker's limited training data.
  • Cohort Models as "Fillers": By pooling data from these acoustically similar "cohort" speakers, a more robust initial model can be created, which, when adapted, better represents the target speaker.
  • Feature-Level vs. Model-Level Cohorts: While some prior work used cohorts at the model level (e.g., merging models), this paper utilizes cohort information at the feature level by using a cohort-derived GMM for adaptation.
  • Test Duration Matters: The paper observes that the effectiveness of the proposed method varies with test utterance duration, suggesting that different classifier structures might be optimal for very short (2 seconds) versus longer test segments.

Key Results

The proposed cohort-UBM approach consistently outperformed the baseline GMM-UBM system across various in-set/out-of-set configurations and test durations.

  • Relative Improvement: The relative improvement in performance (measured by Equal Error Rate - EER) ranged from 10.57% to 58.33%.
  • Specific Improvements:
    • For 2-second test data, relative EER improvements were between 10.96% and 18.18%.
  • For 4-second test data, improvements were more pronounced, ranging from 26.73% to 38.23%.
  • For 6 and 8-second test data, the improvements were even more significant, with relative EER improvements in the range of 30.43% to 58.33%.

Limitations and Future Work

  • Fixed Cohort Size: The current study uses a fixed cohort set size (10 speakers) for all in-set speakers. The authors suggest that performance could be improved if this size were dynamically chosen based on the acoustic similarity scores (e.g., using Eq. 5).
  • Clean Speech Only: The experiments were conducted on clean speech data. The effectiveness of the proposed method needs to be assessed on noisy corpora and under varying channel mismatch conditions.
  • Score Normalization: While the paper uses a world model for score normalization in the baseline, they suggest investigating other score normalization techniques for the proposed method, especially if the development set is not large enough to reliably build a world model.
  • Classifier Structure for Short Utterances: The observation that improvements differ significantly between 2-second and longer test cases suggests that the classifier structure might need to be adapted for very short utterances.

Summary updated

BibTeX
@inproceedings{prakash2006cohort,
  author = {Vinod Prakash and John H. L. Hansen},
  title = {{A cohort - UBM approach to mitigate data sparseness for in-set/out-of-set speaker recognition}},
  booktitle = {Interspeech},
  year = {2006},
}
Figure