The DKU-DukeECE Systems for VoxCeleb Speaker Recognition Challenge 2020

Weiqing Wang, Danwei Cai, Xiaoyi Qin, Ming LiarXiv 2020 · DOI · arXiv

Abstract

In this paper, we present the system submission for the VoxCeleb Speaker Recognition Challenge 2020 (VoxSRC-20) by the DKU-DukeECE team. For track 1, we explore various kinds of state-of-the-art front-end extractors with different pooling layers and objective loss functions. For track 3, we employ an iterative framework for self-supervised speaker representation learning based on a deep neural network (DNN). For track 4, we investigate the whole system pipeline for speaker diarization, including voice activity detection (VAD), uniform segmentation, speaker embedding extraction, and clustering.

Introduction

Speaker recognition "in the wild" remains a formidable challenge due to background noise, overlapping speech, and varying acoustic environments. The VoxCeleb Speaker Recognition Challenge (VoxSRC) tasks researchers with three distinct problems: Speaker Verification (Track 1), Self-Supervised Learning (Track 3), and Speaker Diarization (Track 4). The DKU-DukeECE team aimed to push the boundaries of accuracy in these areas by optimizing neural architectures and refining how models learn from unlabeled data.

Methodology

Track 1: Speaker Verification

The team utilized a fusion of three powerful front-end extractors: ResNet, ResNet-BAM (incorporating a Bottleneck Attention Module), and ECAPA-TDNN (utilizing attentive statistics pooling). To enhance the robustness of similarity scores, Adaptive Symmetric Score Normalization (AS-Norm) was applied to the final outputs.

Track 3: Self-Supervised Learning

The most innovative aspect of the submission is an iterative framework for self-supervised learning. The process begins with Contrastive Self-supervised Learning (CSL) to obtain initial embeddings. These embeddings are then clustered via k-means to generate "pseudo-labels," which are purified by filtering out low-confidence clusters. The network is then retrained using these labels in a supervised fashion, and the process is repeated to bootstrap discriminative power.

Track 4: Speaker Diarization

A comprehensive pipeline was developed, including a ResNet-based Voice Activity Detection (VAD) system, uniform segmentation, and a specialized Attentive vector-to-sequence (Att-v2s) model for similarity measurement. This was followed by spectral clustering and Variational Bayes (VB) resegmentation to refine speaker boundaries.

Key Results and Discussion

Speaker Verification Performance

The fusion of ResNet, ResNet-BAM, and ECAPA-TDNN yielded superior performance compared to individual models. As shown in the performance table below, the fused system achieved an EER of 2.48% on the VoxSRC20-dev set, demonstrating the benefit of combining diverse architectural strengths.

Performance of various front-end extractors and their fusion on VoxCeleb datasets.

Self-Supervised Learning Gains

The iterative pseudo-labeling approach demonstrated significant improvements over the baseline contrastive learning. The EER on the VoxCeleb 1 test set dropped from 8.86% in the initial round to 5.42% by Round 2, representing a performance gain of over 21%. This highlights the power of label purification and iterative retraining in unsupervised contexts.

Performance improvement through iterative rounds of self-supervised learning.

Diarization and Domain Finetuning

For diarization, finetuning the pipeline on the target domain (VoxConverse) was crucial. On the VoxConverse dev-66 set, finetuning reduced the Diarization Error Rate (DER) from 7.53% to 6.46% and the Jaccard Error Rate (JER) from 35.81% to 28.58%.

Conclusion

The DKU-DukeECE team's submission highlights the effectiveness of attention mechanisms, iterative self-supervised learning, and domain-specific finetuning. While the modular diarization pipeline and iterative learning are effective, future work could focus on reducing computational complexity and improving the handling of overlapping speech through end-to-end architectures.

Summary updated

Abstract

In this paper, we present the system submission for the VoxCeleb Speaker Recognition Challenge 2020 (VoxSRC-20) by the DKU-DukeECE team. For track 1, we explore various kinds of state-of-the-art front-end extractors with different pooling layers and objective loss functions. For track 3, we employ an iterative framework for self-supervised speaker representation learning based on a deep neural network (DNN). For track 4, we investigate the whole system pipeline for speaker diarization, including voice activity detection (VAD), uniform segmentation, speaker embedding extraction, and clustering.
BibTeX
@misc{wang2020dku,
  author = {Weiqing Wang and Danwei Cai and Xiaoyi Qin and Ming Li},
  title = {{The DKU-DukeECE Systems for VoxCeleb Speaker Recognition Challenge 2020}},
  year = {2020},
  eprint = {2010.12731},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.12731},
}
Figure