Environment adaptation for robust speaker verification

Kwok-Kwong Yiu, Man‐Wai Mak, Sun‐Yuan Kung2003 · DOI

TL;DR

This paper tackles the challenge of making speaker verification systems reliable over phone lines, where different phone handsets introduce significant distortions. They propose a system that first identifies the handset used and then applies either handset-specific model transformations (MLLR) or uses handset-specific models (PDBNN) to adapt to the acoustic environment, achieving the best performance with MLLR.

What Problem Does This Paper Address?

Speaker verification systems aim to identify individuals based on their voice. However, when these systems are used over public telephone networks, a major hurdle arises: the vast diversity of telephone handsets. Each handset can introduce unique acoustic distortions, leading to a significant drop in recognition accuracy when the verification environment differs from the enrollment environment. This "environmental mismatch" makes robust speaker verification a challenging task, as models trained on clean speech or one type of handset perform poorly on speech from another. The paper seeks to bridge this gap by developing methods to adapt speaker models to these varying acoustic conditions introduced by different phone lines.

Methods

The core of the proposed solution involves a two-pronged approach: handset identification and model adaptation.

  1. Handset Identification: They employ a "handset selector" which uses Gaussian Mixture Models (GMMs). Separate GMMs are trained for each known handset type. During verification, incoming speech is fed to all these GMMs, and a MAXNET selects the most likely handset based on which GMM best matches the speech.
  2. Model Adaptation: Once the handset is identified, the system adapts the speaker and background models in one of two ways:
  3. Probabilistic Decision-Based Neural Networks (PDBNNs): This method uses a two-phase learning rule (locally unsupervised and globally supervised). The globally supervised phase employs gradient descent and reinforced learning to adapt speaker and background models specifically for each handset. This results in handset-dependent models.
  4. Maximum Likelihood Linear Regression (MLLR): This technique estimates adaptation matrices that transform the parameters of existing speaker and background models. These matrices are learned to compensate for the mismatch between enrollment and verification conditions. The paper uses MLLR to transform clean models into handset-specific adapted models.

The system combines these by using the handset selector to choose the appropriate adaptation strategy (either PDBNN-adapted models or MLLR-transformed models) for the identified handset.

Datasets

  • HTIMIT Corpus: This corpus was used for both training and evaluation. It was created by playing gender-balanced TIMIT corpus speech through 9 different telephone handsets and a Sennheiser head-mounted microphone. This dataset is specifically designed to study handset variability in speech and speaker recognition.

Key Insights and Ideas

  • Handset Identification is Crucial: The paper strongly suggests that explicitly identifying the handset type before adaptation is a key step towards robust speaker verification. This allows for more targeted and effective adaptation.
  • MLLR Outperforms PDBNN for Adaptation: While both PDBNN and MLLR showed improvements over baseline methods, MLLR adaptation achieved the best performance. This is attributed to MLLR's ability to find global transformation matrices with less adaptation data compared to PDBNN's need for more data for its discriminative training.
  • Model-Based Adaptation is Effective: The study highlights that modifying model parameters (as done by MLLR and PDBNN) is a powerful strategy for compensating for acoustic mismatches, often outperforming feature-based transformations.
  • Trade-off Between Data and Complexity: PDBNNs, with their reinforced learning, are powerful but require more data and potentially more retraining for new speakers. MLLR, while needing more data for robust matrix estimation, offers a more straightforward adaptation process once matrices are learned.

Key Results

  • The proposed system, combining handset identification with MLLR adaptation, achieved the lowest error rate among all tested methods.
  • Both MLLR and PDBNN adaptation significantly reduced error rates compared to classical methods like Cepstral Mean Subtraction (CMS), Handset Normalization (Hnorm), and Test Normalization (Tnorm).
  • MLLR adaptation, particularly when using more speakers for training the adaptation matrices (MLLR-100), showed superior performance over other methods, achieving an average equal error rate of 6.67%.
  • Stochastic Feature Transformation (SFT) performed comparably to MLLR when using limited training data (SFT-20 vs. MLLR-20), but MLLR's performance scaled better with more data.

Limitations and Future Work

  • PDBNN Data Requirements: The paper notes that PDBNN adaptation's performance is limited by insufficient adaptation data, suggesting it requires more data to be effective than MLLR.
  • MLLR Data Requirements: While MLLR performs well, it requires more data to robustly estimate global transformation matrices.
  • New Speaker Adaptation: The paper mentions that PDBNNs require additional training for new speakers, whereas MLLR adaptation is more streamlined once matrices are estimated.
  • Future Work: The paper implicitly suggests further exploration into optimizing the balance between adaptation data requirements and performance for different techniques, and potentially combining the strengths of both PDBNN and MLLR.

Summary updated

Abstract

In speaker verification over public telephone networks, utterances can be obtained from different types of handsets. Different handsets may introduce different degrees of distortion to the speech signals. This paper attempts to combine a handset selector with (1) handset-specific transformations and (2) handset-dependent speaker models to reduce the effect caused by the acoustic distortion. Specifically, a number of Gaussian mixture models are independently trained to identify the most likely handset given a test utterance; then during recognition, the speaker model and background model are either transformed by MLLR-based handset-specific transformation or respectively replaced by a handset-dependent speaker model and a handset-dependent background model whose parameters were adapted by reinforced learning to fit the new environment. Experimental results based on 150 speakers of the HTIMIT corpus show that environment adaptation based on both MLLR and reinforced learning outperforms the classical CMS, Hnorm and Tnorm approaches, with MLLR adaptation achieves the best performance.
BibTeX
@misc{yiu2003environment,
  author = {Kwok-Kwong Yiu and Man‐Wai Mak and Sun‐Yuan Kung},
  title = {{Environment adaptation for robust speaker verification}},
  year = {2003},
  doi = {10.21437/eurospeech.2003-762},
}
Figure