i-Vector DNN Scoring and Calibration for Noise Robust Speaker Verification

Zhili Tan, Man‐Wai Mak2017 · DOI

TL;DR

This paper introduces a multi-task deep neural network (DNN) approach to improve speaker verification systems in noisy environments. By training a single DNN to simultaneously predict score shifts, signal-to-noise ratios (SNRs), and utterance durations, the system achieves more robust calibration of traditional PLDA scores. The research also explores using DNNs to replace PLDA entirely, finding that while DNNs can estimate score shifts effectively, PLDA remains crucial for overall performance.

What Problem Does This Paper Address?

Speaker verification systems, which aim to identify individuals based on their voice, often struggle in noisy environments. Traditional methods like i-vectors combined with Probabilistic Linear Discriminant Analysis (PLDA) are powerful but can be sensitive to variations in background noise and utterance duration. These variations can lead to "score shifts," where the system's confidence in a match is artificially inflated or deflated. While previous work has focused on calibrating these scores or enhancing features, this paper tackles the challenge of directly learning to correct these score shifts using DNNs, specifically by leveraging multi-task learning to improve the DNN's ability to learn the primary calibration task.

Methods

The core of this paper is a multi-task DNN designed for score calibration. Unlike single-task DNNs that focus on one objective, this network is trained to perform several tasks simultaneously:

  • Main Task (Calibration):
    • Predicting the score shift needed to correct the noisy PLDA score.
  • Predicting the clean PLDA score directly.
  • Auxiliary Tasks:
    • Predicting the Signal-to-Noise Ratio (SNR) of the test and target utterances.
  • Predicting the duration of the test and target utterances.
  • Classifying whether an i-vector pair belongs to the same speaker or a different speaker.

The DNN takes i-vector pairs and the uncalibrated PLDA score as input. The auxiliary tasks are hypothesized to provide valuable "inductive bias," helping the DNN learn the primary calibration task more effectively. The paper also investigates a "DNN Scoring Machine" that attempts to perform calibration without relying on the noisy PLDA score as input, aiming to replace the PLDA model entirely.

Datasets

The experiments were conducted using data from the NIST 2012 Speaker Recognition Evaluation (SRE), specifically the Common Condition 4 (CC4, male) subset. Development data was sourced from NIST 2005-2010 SREs. To simulate noisy conditions, babble noise was added to the target and test utterances at various Signal-to-Noise Ratios (SNRs): 15dB, 6dB, and 0dB.

Key Insights and Ideas

  • Auxiliary Tasks are Crucial: The inclusion of auxiliary tasks (SNR, duration, speaker identity classification) significantly improves the DNN's ability to learn the main calibration task. Without these, a single-task DNN struggles to accurately recover clean scores.
  • Uncalibrated PLDA Score is Essential: The noisy PLDA score itself is a vital input for the DNN. When this input is removed (in the "DNN Scoring Machine" experiment), the DNN can still estimate score shifts but cannot reliably predict clean scores. This highlights the indispensable role of the PLDA model, even when using DNNs for calibration.
  • Classification Task Aids Convergence: While initially leading to higher test errors, the classification task (predicting same/different speaker) ultimately guides the DNN towards a better solution, improving the accuracy of recovered clean scores over time.
  • Posterior Odds as Scores: Using the posterior odds derived from the DNN's classification outputs can serve as effective verification scores, outperforming the baseline in many noisy conditions.

Key Results

  • Improved Calibration: The proposed multi-task DNN calibration methods consistently outperform the baseline (no calibration) and a simple SNR-dependent score shift method, especially in highly noisy conditions (0dB SNR). For instance, in Table 2, the "Recover Clean Scores by DNN (Eq. 6)" method achieves an EER of 1.54% at 0dB, compared to 1.68% for the baseline.
  • DNN Scoring Machine Limitations: When the noisy PLDA score is removed as input, the DNN scoring machine shows limited success. While it can estimate score shifts effectively (Row 3 in Table 3), it struggles to recover clean scores (Row 2) or produce reliable posterior odds (Row 4), indicating that the PLDA model is still necessary for optimal performance.
  • Auxiliary Task Contributions: Adding SNR information to the auxiliary tasks significantly improves robustness. Duration information, however, showed a slight performance degradation when added.

Limitations and Future Work

  • PLDA Indispensability: The paper concludes that while DNNs can significantly enhance speaker verification in noise, the PLDA model remains indispensable for achieving the best performance, particularly when aiming to recover clean scores. The "DNN Scoring Machine" that attempts to replace PLDA shows limitations.
  • Future Directions: The authors suggest that future work could explore further optimizing the multi-task DNN architecture and training strategies. Investigating other auxiliary tasks or different ways to integrate PLDA outputs with DNN predictions could also be promising avenues. The paper also implicitly suggests that further research into DNNs that can fully replace PLDA would be valuable, though current results indicate this is a challenging goal.

Summary updated

Abstract

This paper proposes applying multi-task learning to train deep neural networks (DNNs) for calibrating the PLDA scores of speaker verification systems under noisy environments. To facilitate the DNNs to learn the main task (calibration), several auxiliary tasks were introduced, including the prediction of SNR and duration from i-vectors and classifying whether an i-vector pair belongs to the same speaker or not. The possibility of replacing the PLDA model by a DNN during the scoring stage is also explored. Evaluations on noise contaminated speech suggest that the auxiliary tasks are important for the DNNs to learn the main calibration task and that the uncalibrated PLDA scores are an essential input to the DNNs. Without this input, the DNNs can only predict the score shifts accurately, suggesting that the PLDA model is indispensable.
BibTeX
@misc{tan2017i,
  author = {Zhili Tan and Man‐Wai Mak},
  title = {{i-Vector DNN Scoring and Calibration for Noise Robust Speaker Verification}},
  year = {2017},
  doi = {10.21437/interspeech.2017-656},
}

Loading related content...

Figure