SJTU-AISPEECH System for VoxCeleb Speaker Recognition Challenge 2022

Zhengyang Chen, Bing Han, Xu Xiang, Houjun Huang, Bei Liu, Yanmin QianarXiv 2022 · DOI · arXiv

Abstract

The SJTU-AISPEECH system for the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC 2022) focuses on addressing scale and domain shift in speaker recognition. By combining high-performance ResNet and Res2Net backbones with advanced domain adaptation strategies, the team secured 3rd place in Track 1 (Closed-set Supervised) and 4th place in Track 3 (Semi-supervised Domain Adaptation). A core contribution is the Dynamic Loss-gate and Label Correction (DLG-LC) mechanism, which enhances robust semi-supervised learning when transitioning from Western (VoxCeleb) to Chinese (CN-Celeb) speech domains.

Introduction

Speaker recognition systems face significant challenges regarding data scale and domain shift. While models trained on massive datasets like VoxCeleb2 achieve high accuracy, their performance often degrades when applied to different languages or acoustic environments. This work addresses two tracks of VoxSRC 2022: Track 1, focused on supervised learning within the VoxCeleb domain, and Track 3, focused on adapting models from VoxCeleb (Source) to CN-Celeb (Target) using unlabeled target data.

Methodology

Architecture and Training

The system utilizes various ResNet and Res2Net architectures with different base channel widths (c32 and c64). Two distinct system types were developed: Online systems using on-the-fly data augmentation and Offline systems using pre-processed, heavily augmented datasets.

For pooling, the team compared standard Statistic Pooling with Multi-Query Multi-Head Attention (MQMHA). To maximize speaker discriminability, the training objective combined Additive Angular Margin (AAM) softmax, K-subcenter loss (to handle multiple centers per speaker), and Inter-TopK loss.

Table 1: Configuration of Online and Offline systems, including backbones and pooling methods.

Domain Adaptation (Track 3)

To bridge the gap between the source (VoxCeleb) and target (CN-Celeb) domains, several adaptation techniques were explored:

  • Statistic Adaptation (SA): Normalizing embeddings using target domain statistics.
  • One-head Classification Loss (OCL): Forcing the network to map both domains into a shared discriminative embedding space by using a single output layer for both source and target data.
  • Dynamic Loss-gate and Label Correction (DLG-LC): A filtering mechanism to handle noisy pseudo-labels in the unlabeled target set. By modeling the loss distribution as a Gaussian, the system identifies and ignores unreliable samples, preventing overfitting to noise.

Datasets

The primary supervised training set was VoxCeleb2 (Dev), containing approximately 1.1 million utterances from 5,994 speakers. For Track 3, the target domain was CN-Celeb/CN-Celeb2, which includes a large unlabeled set (~455k utterances) and a small labeled validation set of 50 speakers. Data augmentation for offline systems utilized the MUSAN and RIR datasets to increase training volume by 15x.

Results and Discussion

In Track 1, the fusion of multiple models achieved an Equal Error Rate (EER) of 1.056% on the validation set and 1.911% on the test set.

For Track 3, the combination of OCL and DLG-LC provided substantial improvements. Statistic Adaptation alone reduced the EER from 14.6% to 11.65%. The final fusion system reached an EER of 7.135% on the validation set.

Table 2: Performance comparison of different adaptation methods on the Track 3 validation set.

The results highlight that while deeper backbones (e.g., ResNet221) generally perform better, the gains plateau. The OCL approach consistently outperformed two-head classification, and the DLG-LC strategy was vital for managing the "double-edged sword" of pseudo-labeling.

Conclusion

The SJTU-AISPEECH system demonstrates that combining diverse architectures with sophisticated domain adaptation and noise-robust semi-supervised learning is highly effective for large-scale speaker recognition. Future work will focus on improving the quality of initial pseudo-labels through iterative self-training or self-supervised pre-training methods tailored for speech.

Summary updated

Abstract

This report describes the SJTU-AISPEECH system for the Voxceleb Speaker Recognition Challenge 2022. For track1, we implemented two kinds of systems, the online system and the offline system. Different ResNet-based backbones and loss functions are explored. Our final fusion system achieved 3rd place in track1. For track3, we implemented statistic adaptation and jointly training based domain adaptation. In the jointly training based domain adaptation, we jointly trained the source and target domain dataset with different training objectives to do the domain adaptation. We explored two different training objectives for target domain data, self-supervised learning based angular proto-typical loss and semi-supervised learning based classification loss with estimated pseudo labels. Besides, we used the dynamic loss-gate and label correction (DLG-LC) strategy to improve the quality of pseudo labels when the target domain objective is a classification loss. Our final fusion system achieved 4th place (very close to 3rd place, relatively less than 1%) in track3.
BibTeX
@misc{chen2022sjtu,
  author = {Zhengyang Chen and Bing Han and Xu Xiang and Houjun Huang and Bei Liu and Yanmin Qian},
  title = {{SJTU-AISPEECH System for VoxCeleb Speaker Recognition Challenge 2022}},
  year = {2022},
  eprint = {2209.09076},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.09076},
}
Figure