Abstract
In this paper, we present the system submission for the VoxCeleb Speaker Recognition Challenge 2020 (VoxSRC-20) by the DKU-DukeECE team. For track 1, we explore various kinds of state-of-the-art front-end extractors with different pooling layers and objective loss functions. For track 3, we employ an iterative framework for self-supervised speaker representation learning based on a deep neural network (DNN). For track 4, we investigate the whole system pipeline for speaker diarization, including voice activity detection (VAD), uniform segmentation, speaker embedding extraction, and clustering.
Introduction
Speaker recognition "in the wild" remains a formidable challenge due to background noise, overlapping speech, and varying acoustic environments. The VoxCeleb Speaker Recognition Challenge (VoxSRC) tasks researchers with three distinct problems: Speaker Verification (Track 1), Self-Supervised Learning (Track 3), and Speaker Diarization (Track 4). The DKU-DukeECE team aimed to push the boundaries of accuracy in these areas by optimizing neural architectures and refining how models learn from unlabeled data.
Methodology
Track 1: Speaker Verification
The team utilized a fusion of three powerful front-end extractors: ResNet, ResNet-BAM (incorporating a Bottleneck Attention Module), and ECAPA-TDNN (utilizing attentive statistics pooling). To enhance the robustness of similarity scores, Adaptive Symmetric Score Normalization (AS-Norm) was applied to the final outputs.
Track 3: Self-Supervised Learning
The most innovative aspect of the submission is an iterative framework for self-supervised learning. The process begins with Contrastive Self-supervised Learning (CSL) to obtain initial embeddings. These embeddings are then clustered via k-means to generate "pseudo-labels," which are purified by filtering out low-confidence clusters. The network is then retrained using these labels in a supervised fashion, and the process is repeated to bootstrap discriminative power.
Track 4: Speaker Diarization
A comprehensive pipeline was developed, including a ResNet-based Voice Activity Detection (VAD) system, uniform segmentation, and a specialized Attentive vector-to-sequence (Att-v2s) model for similarity measurement. This was followed by spectral clustering and Variational Bayes (VB) resegmentation to refine speaker boundaries.
Key Results and Discussion
Speaker Verification Performance
The fusion of ResNet, ResNet-BAM, and ECAPA-TDNN yielded superior performance compared to individual models. As shown in the performance table below, the fused system achieved an EER of 2.48% on the VoxSRC20-dev set, demonstrating the benefit of combining diverse architectural strengths.
Self-Supervised Learning Gains
The iterative pseudo-labeling approach demonstrated significant improvements over the baseline contrastive learning. The EER on the VoxCeleb 1 test set dropped from 8.86% in the initial round to 5.42% by Round 2, representing a performance gain of over 21%. This highlights the power of label purification and iterative retraining in unsupervised contexts.
Diarization and Domain Finetuning
For diarization, finetuning the pipeline on the target domain (VoxConverse) was crucial. On the VoxConverse dev-66 set, finetuning reduced the Diarization Error Rate (DER) from 7.53% to 6.46% and the Jaccard Error Rate (JER) from 35.81% to 28.58%.
Conclusion
The DKU-DukeECE team's submission highlights the effectiveness of attention mechanisms, iterative self-supervised learning, and domain-specific finetuning. While the modular diarization pipeline and iterative learning are effective, future work could focus on reducing computational complexity and improving the handling of overlapping speech through end-to-end architectures.
Summary updated