The JHU Speaker Recognition System for the VOiCES 2019 Challenge

David Snyder, Jesús Villalba, Nanxin Chen, Daniel Povey, Gregory Sell, Najim Dehak, Sanjeev KhudanpurInterspeech 2019 · OpenReview

Abstract

This paper describes the systems developed by the JHU team for the speaker recognition track of the 2019 VOiCES from a Distance Challenge. On this far-field task, we achieved good performance using systems based on state-of-the-art deep neural network (DNN) embeddings. In this paradigm, a DNN maps variable-length speech segments to speaker embeddings, called x-vectors, that are then classified using probabilistic linear discriminant analysis (PLDA). Our submissions were composed of three x-vector-based systems that differed primarily in the DNN architecture, temporal pooling mechanism, and training objective function. On the evaluation set, our best single-system submission used an extended time-delay architecture, and achieved 0.435 in actual DCF, the primary evaluation metric. A fusion of all three x-vector systems was our primary submission, and it obtained an actual DCF of 0.362.

Introduction

Traditional speaker recognition systems often struggle in "complex environmental settings" where the speaker is far from the microphone (far-field) and background noise—such as TV, music, or babble—combined with room reverberation distorts the speech signal. The VOiCES (Voices Obscured in Complex Environmental Settings) Challenge was established to benchmark systems against these realistic hurdles. The JHU team participated in the "fixed" condition, requiring architectural innovation and robust data augmentation within a constrained set of training data.

Methods and Architectures

The JHU system utilizes the x-vector paradigm, where a DNN maps variable-length audio segments into fixed-length embeddings representing speaker identity. These embeddings are subsequently compared using Probabilistic Linear Discriminant Analysis (PLDA).

The team developed three primary architectures:

  1. Extended TDNN (etdnn): An evolution of the standard Time-Delay Neural Network using a wider temporal context and interleaved dense layers to increase capacity.
  2. Factorized TDNN (ftdnn): This architecture reduces parameters by factorizing weight matrices into low-rank products and incorporates skip connections to facilitate deeper network training.
  3. ResNet34-LDE: A 2D residual network that replaces standard statistics pooling with Learnable Dictionary Encoding (LDE). LDE clusters frame-level features into multiple centers, better capturing speech sound distributions in reverberant environments.
The architecture of the Factorized TDNN (ftdnn) system, highlighting the use of skip connections and factorized layers.

For the back-end, Linear Discriminant Analysis (LDA) was used for dimensionality reduction, followed by length normalization and PLDA scoring.

Datasets and Augmentation

Training data included VoxCeleb-1, VoxCeleb-2, and Speakers in the Wild (SITW). Data augmentation was critical; the team utilized the MUSAN corpus for noise and music, the AIR dataset for simulated reverberation, and a custom "Babble" noise set. Evaluation was performed on the VOiCES 2019 Dev and Eval sets, featuring Librispeech audio recorded in physical rooms with varying microphone distances.

Key Results and Insights

The primary evaluation metric was the Actual Detection Cost Function (Act DCF). While the Extended TDNN was a strong single system, a fusion of all three architectures provided a significant 17% improvement in Act DCF on the development set, demonstrating the benefit of model diversity.

Performance metrics (EER and DCF) for the JHU systems across different noise conditions, microphone positions, and adaptation techniques.

Key Findings:

  • The Impact of Fusion: The primary submission, a fusion of all three systems, achieved an Act DCF of 0.362 on the Eval set, outperforming any individual model.
  • Environmental Sensitivity: Performance varied significantly by condition. "Babble" noise proved most detrimental to accuracy. Additionally, performance degraded as microphone distance increased, with the "wall (fully obstructed)" position causing the highest error rates.
  • Domain Mismatch: A substantial performance gap existed between the Dev set (~1% EER) and the Eval set (~5% EER), likely due to more challenging room geometries and closer proximity of noise sources in the evaluation environment.

Conclusion

The JHU VOiCES 2019 system highlights the effectiveness of ensemble methods and specialized architectures like F-TDNN and ResNet-LDE for far-field speaker recognition. Future work will focus on improving domain adaptation to bridge the gap between training and real-world evaluation environments and addressing catastrophic failures in obstructed microphone scenarios.

Summary updated

Abstract

This paper describes the systems developed by the JHU team for the speaker recognition track of the 2019 VOiCES from a Distance Challenge. On this far-field task, we achieved good performance using systems based on state-of-the-art deep neural network (DNN) embeddings. In this paradigm, a DNN maps variable-length speech segments to speaker embeddings, called x-vectors, that are then classified using probabilistic linear discriminant analysis (PLDA). Our submissions were composed of three x-vector-based systems that differed primarily in the DNN architecture, temporal pooling mechanism, and training objective function. On the evaluation set, our best single-system submission used an extended time-delay architecture, and achieved 0.435 in actual DCF, the primary evaluation metric. A fusion of all three x-vector systems was our primary submission, and it obtained an actual DCF of 0.362.
BibTeX
@inproceedings{snyder2019jhu,
  author = {David Snyder and Jesús Villalba and Nanxin Chen and Daniel Povey and Gregory Sell and Najim Dehak and Sanjeev Khudanpur},
  title = {{The JHU Speaker Recognition System for the VOiCES 2019 Challenge}},
  booktitle = {Interspeech},
  year = {2019},
  url = {https://openreview.net/forum?id=iRfO69tqmZ},
}
Figure