The DKU-SMIIP System for NIST 2018 Speaker Recognition Evaluation

Danwei Cai, Weicheng Cai, Ming LiInterspeech 2019

TL;DR

This paper presents the DKU-SMIIP system for the NIST SRE 2018 speaker recognition challenge, exploring various state-of-the-art front-end and back-end techniques for text-independent speaker verification. They achieved competitive results by combining multiple embedding extraction methods and advanced back-end modeling for domain adaptation and variability compensation.

What Problem Does This Paper Address?

Speaker recognition, particularly text-independent speaker verification (TISV), aims to identify individuals based on their voice, regardless of what they are saying. The NIST SRE challenges are crucial benchmarks for advancing this field. The NIST SRE 2018 specifically focused on TISV with two challenging tasks: CMN2 (non-English conversational telephone speech with channel, language, and duration mismatches) and VAST (English audio from YouTube with complex environments, reverberation, noise, and potential multi-talker scenarios). These real-world conditions introduce significant mismatches between training and testing data, making robust speaker recognition a difficult problem. The paper addresses the challenge of building a high-performing TISV system that can effectively handle these various mismatches.

Methods

The DKU-SMIIP system employs a multi-pronged approach, combining several state-of-the-art techniques for both feature extraction (front-end) and classification (back-end).

Front-end Extractors: * MFCC i-vector: A classic approach using Mel-Frequency Cepstral Coefficients (MFCCs) and i-vectors for speaker representation. * DNN tandem i-vector: Combines phonetic-aware features extracted from a Deep Neural Network (DNN) with acoustic MFCC features. * TDNN x-vector: Leverages Time Delay Neural Networks (TDNNs) to learn speaker embeddings from frame-level features, pooling temporal statistics. * Deep ResNet: Utilizes a deep Residual Neural Network (ResNet) architecture, exploring variations in network width, encoding layer designs (like Global Average Pooling - GAP, and Learnable Dictionary Encoding - LDE), and loss functions (Softmax vs. A-softmax).

Back-end Modeling: After extracting speaker embeddings, the system applies several techniques to improve robustness:

  • Variabilities Compensation: Linear Discriminant Analysis (LDA) or Locality Sensitive Discriminant Analysis (LSDA) are used to select speaker-relevant features and reduce irrelevant variability.
  • Domain Adaptation: Correlation Alignment (CORAL) is employed to align feature distributions between out-of-domain (training) and in-domain (testing) data.
  • In-domain Whitening: Applied before PLDA to further reduce variabilities by whitening test data using in-domain statistics.
  • Gaussian PLDA: Probabilistic Linear Discriminant Analysis (PLDA) is used for scoring, with a full covariance residual noise term and full-rank eigenvoice subspace.
  • Cosine Similarity: Used as an alternative scoring method, particularly for ResNet-based systems.
  • Score Normalization: Adaptive Symmetric Score Normalization (AS-Norm) is applied to normalize scores.
  • System Fusion: Multiple subsystem scores are fused using logistic regression to learn optimal scales and biases for each system, followed by an equal-weighted sum.

Datasets

  • Training Data: A large corpus including SRE04-16, MIXER 6, Switchboard, VoxCeleb1, and VoxCeleb2, totaling 14,467 speakers. Data augmentation with additive noises (MUSAN dataset) and reverberation (simulated RIRs) was also applied.
  • Evaluation Data: NIST SRE 2018 CMN2 and VAST tasks.
  • Score Normalization Corpora: SRE18 unlabeled, SRE16 unlabeled (for CMN2), and SITW (Speaker In The Wild) dataset (for VAST).

Key Insights and Ideas

  • Fusion is Key: Combining multiple diverse front-end extractors and back-end models significantly improves performance, demonstrating the complementary nature of different approaches.
  • Deep Learning Dominance: Deep learning-based methods, particularly TDNN x-vectors and ResNets, generally outperform traditional i-vector approaches, especially in challenging mismatched conditions.
  • ResNet Potential: The deep ResNet architecture shows strong potential, with further improvements observed by exploring different encoding layer designs (e.g., LDE) and advanced loss functions (e.g., A-softmax).
  • Domain Adaptation Matters: Techniques like CORAL and in-domain whitening are crucial for bridging the gap between training and testing data in mismatched scenarios.
  • Subtleties in Score Normalization: The choice of score normalization strategy (e.g., AS-Norm1 vs. AS-Norm2) can have a significant impact, and what works optimally on a development set might not generalize perfectly to the evaluation set.

Key Results

  • The submitted system achieved an actual detection cost (actC) of 0.392 on CMN2 and 0.494 on VAST for the fixed condition.
  • The fused system achieved the best performance, with a minC of 0.324 and actC of 0.329 on CMN2 development, and minC of 0.392 and actC of 0.409 on CMN2 evaluation.
  • On the VAST task, the fused system achieved a minC of 0.259 and actC of 0.259 on development, and minC of 0.494 and actC of 0.515 on evaluation.
  • Post-evaluation experiments showed that the LDE layer and A-softmax loss further improved the ResNet system's performance, particularly in language-mismatched conditions. For instance, a ResNet v2 system with LDE and A-softmax achieved a minC of 0.300 on VAST development.

Limitations and Future Work

  • Data Mismatch: While various techniques were employed, significant mismatches between training and testing data remain a core challenge.
  • VAST Development Set Size: The small size of the VAST development set made it difficult to reliably tune parameters, leading to some unexpected results on the evaluation set.
  • Future Work: The authors suggest further exploration of encoding layer designs and loss functions for deep ResNet systems. They also highlight the potential of LDE layer and A-softmax loss for improving performance in challenging scenarios. The paper implies that continued research into robust domain adaptation and fusion strategies will be beneficial.

Summary updated

BibTeX
@inproceedings{cai2019dku,
  author = {Danwei Cai and Weicheng Cai and Ming Li},
  title = {{The DKU-SMIIP System for NIST 2018 Speaker Recognition Evaluation}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure