Variational Domain Adversarial Learning for Speaker Verification

Youzhi Tu, Man‐Wai Mak, Jen‐Tzung Chien2019 · DOI

TL;DR

This paper introduces Variational Domain Adversarial Learning (VDANN), a novel neural network architecture that tackles the challenge of domain mismatch in speaker verification. By combining domain adversarial training with variational autoencoders, VDANN learns speaker-invariant and Gaussian-distributed features, significantly improving performance on mismatched datasets.

What Problem Does This Paper Address?

Speaker verification systems aim to identify individuals based on their voice. Ideally, the data used to train these systems should match the data they encounter in real-world scenarios (i.e., be "in-domain"). However, in practice, training data often comes from different sources or conditions than the test data, leading to "domain mismatch." This mismatch degrades performance because the system has learned patterns specific to the training domain that don't generalize well. This paper focuses on unsupervised domain adaptation, where we have a large amount of labeled source data and only unlabeled target data, to bridge this gap and make speaker verification systems more robust to varying acoustic conditions and data sources.

Methods

The core innovation is the Variational Domain Adversarial Neural Network (VDANN). It builds upon the Domain Adversarial Neural Network (DANN), which uses adversarial training to learn features that are both speaker-discriminative and domain-invariant. DANN achieves this by training a feature extractor (encoder) to fool a domain classifier, while simultaneously training a speaker classifier to correctly identify speakers from these extracted features.

VDANN enhances DANN by incorporating a Variational Autoencoder (VAE). The VAE's role is to impose a variational regularization on the learned features. Specifically, it encourages the encoder to produce latent representations that follow a Gaussian distribution. This is crucial because standard Probabilistic Linear Discriminant Analysis (PLDA) models, commonly used in speaker verification backends, assume Gaussian-distributed embeddings. By ensuring the features are Gaussian, VDANN makes them directly compatible with PLDA, avoiding the need for more complex or less optimal solutions like heavy-tailed PLDA or i-vector length normalization.

The VDANN architecture consists of:

  • Encoder (E): Extracts features from input audio.
  • Decoder (G): Reconstructs the input from the latent features (part of the VAE).
  • Speaker Classifier (C): Identifies the speaker from the extracted features.
  • Domain Classifier (D): Tries to distinguish the domain of the extracted features.

The training objective is a combination of:

  • Speaker classification loss ($L_c$): To ensure speaker discriminability.
  • Domain classification loss ($L_D$): Minimized by the domain classifier, but its gradient is reversed during feature extractor training to achieve domain invariance.
  • Variational Autoencoder loss ($L_{VAE}$): To enforce Gaussianity on the latent features.

Datasets

The paper uses data from four domains for training the VDANN and DANN models:

  • SRE04-10: A collection of NIST Speaker Recognition Evaluation datasets.
  • Voxceleb1: A large-scale speaker recognition dataset.
  • SwitchBoard II: A dataset containing telephone conversations.
  • SITW (Speaker In The Wild): A dataset of in-the-wild speech recordings.

Evaluation is performed on the NIST SRE16 and SRE18-CMN2 datasets, which are known for their domain mismatch challenges.

Key Insights and Ideas

  • Gaussianity is Key for PLDA: The paper strongly emphasizes that for standard PLDA models to perform optimally, speaker embeddings need to follow a Gaussian distribution. Adversarial training alone doesn't guarantee this.
  • VAE as a Gaussian Regularizer: The VAE's KL-divergence term acts as a powerful regularizer, effectively pushing the learned latent representations towards a desired Gaussian distribution without sacrificing domain invariance or speaker discriminability.
  • VDANN Outperforms DANN: The experimental results clearly show that VDANN consistently outperforms the standard DANN, highlighting the benefit of explicitly enforcing Gaussianity.
  • Domain Adaptation Benefits: The VDANN approach significantly reduces domain mismatch, leading to improved performance metrics like Equal Error Rate (EER) and minimum Detection Cost Function (minDCF).

Key Results

  • On SRE16, VDANN achieved an EER of 11.17% and a minDCF of 0.798 without PLDA adaptation, outperforming the baseline (11.30% EER, 0.890 minDCF) and DANN (11.62% EER, 0.822 minDCF).
  • With PLDA adaptation, VDANN achieved 8.21% EER and 0.584 minDCF, again outperforming the baseline and DANN.
  • Similar improvements were observed on SRE18-CMN2, with VDANN achieving 10.25% EER and 0.667 minDCF without PLDA adaptation.
  • Statistical significance tests (McNemar's test) confirmed that the improvements of VDANN over DANN were statistically significant.
  • Q-Q plots visually demonstrated that VDANN-transformed x-vectors were closer to a Gaussian distribution compared to DANN-transformed vectors and the original x-vectors.

Limitations and Future Work

  • Kaldi PLDA Adaptation: While VDANN shows strong results, the paper notes that when combined with Kaldi's powerful PLDA adaptation, the performance gains over the baseline are smaller, suggesting that PLDA adaptation itself is a very effective technique. However, VDANN still provides a benefit even in this scenario.
  • Hyperparameter Tuning: The paper mentions setting hyperparameters $\alpha$ and $\beta$ to 0.1 for VDANN. Further exploration of optimal hyperparameter settings could potentially yield even better results.
  • Future Work: The authors suggest that VDANN's ability to produce Gaussian-distributed, domain-invariant, and speaker-discriminative features makes it a promising direction for future research in robust speaker verification. The paper implicitly suggests exploring VDANN in other domain adaptation scenarios or with different backend models.

Summary updated

Abstract

20th Annual Conference of the International Speech Communication Association: Crossroads of Speech and Language, INTERSPEECH 2019, Graz, Austria, 15-19 September 2019
BibTeX
@misc{tu2019variational,
  author = {Youzhi Tu and Man‐Wai Mak and Jen‐Tzung Chien},
  title = {{Variational Domain Adversarial Learning for Speaker Verification}},
  year = {2019},
  doi = {10.21437/interspeech.2019-2168},
}
Figure