Cross-Scene Speaker Verification Based on Dynamic Convolution for the CNSRC 2022 Challenge

Jialin Zhang, Qinghua Ren, Youcai Qin, Zikai Wan, Qirong Mao2022 · DOI

TL;DR

This paper tackles the challenge of speaker verification in the presence of diverse and noisy audio conditions, particularly for Chinese speakers. The authors introduce a novel "Dynet-ECAPA-TDNN" model that leverages dynamic convolution for more robust feature extraction and a gradient reversal layer to mitigate the negative impact of scene information, achieving state-of-the-art results on the CNSRC 2022 challenge.

What Problem Does This Paper Address?

Speaker verification, the task of identifying who is speaking, is crucial for many security and convenience applications. However, real-world scenarios are fraught with challenges like background noise, varying acoustic environments (scenes), and different speaking styles. The CNSRC 2022 challenge, focusing on Chinese celebrity speech (CN-Celeb datasets), presents a particularly difficult problem due to its complex conditions and the significant impact of "scene mismatch" – where the acoustic environment of the enrollment audio differs from the test audio. This mismatch can drastically reduce the accuracy of even advanced speaker verification systems. Existing methods, while effective on cleaner datasets like VoxCeleb, struggle to generalize well to these more challenging, real-world conditions.

Methods

The core of the proposed solution lies in two main innovations:

  • Dynet-ECAPA-TDNN: The authors adapt the successful ECAPA-TDNN architecture by replacing its Res2Net module with a "dynamic convolution" module. Unlike standard convolutions that use fixed parameters, dynamic convolution dynamically generates convolution kernels based on the input audio. This allows the model to adapt its feature extraction to the specific characteristics of each utterance, leading to more robust representations. They explored different numbers of dynamic kernels (K=4 and K=16) to find the optimal balance.
  • Gradient Reversal Layer (GRL) for Feature Decoupling: To combat the negative influence of scene information, the researchers employ a gradient reversal layer. This technique is used in conjunction with an auxiliary task (scene classification). During training, the GRL reverses the gradient signal from the scene classifier, effectively pushing the speaker embedding to become invariant to scene-specific features. This aims to decouple speaker identity from scene information, making the system more robust to scene mismatch.

For the open track, they utilized a pre-trained model on the larger VoxCeleb dataset and then fine-tuned it on the CN-Celeb datasets.

Datasets

  • CN-Celeb1 & CN-Celeb2: These are the primary datasets for the CNSRC 2022 challenge. They are characterized by unconstrained recording conditions, diverse speech genres, and scene labels. CN-Celeb1 contains about 1,000 speakers and 130,000 utterances, while CN-Celeb2 is larger with 2,000 speakers and 520,000 utterances. They are noted to be more challenging than VoxCeleb due to their real-world nature.
  • RIRS_NOISE: This dataset was used for data augmentation, specifically to add simulated and real indoor reverberation and noise to the training data, enhancing the model's robustness.
  • VoxCeleb: Used for pre-training models in the open track, this large-scale English speaker recognition dataset provides a strong foundation before fine-tuning on the Chinese datasets.

Key Insights and Ideas

  • Dynamic Convolution is Key for Complex Scenes: The paper strongly suggests that static convolution kernels are insufficient for handling the intricate and varied acoustic conditions found in datasets like CN-Celeb. Dynamic convolution's ability to adapt kernel parameters on-the-fly is crucial for extracting discriminative speaker features in such environments.
  • Scene Information is a Major Hindrance: The experiments clearly demonstrate that scene information, when mismatched between enrollment and test, significantly degrades speaker verification performance. The GRL approach, while not perfectly solving the issue, shows promise in mitigating this negative impact.
  • Two-Stage Training is Beneficial: Employing a two-stage training strategy, starting with shorter utterances and then fine-tuning with longer ones, proved to be an effective way to leverage the available data and improve performance.
  • Score Fusion is a Powerful Ensemble Method: Combining the outputs of multiple well-performing systems through score fusion was a critical step in achieving the final top results.

Key Results

The proposed methods achieved impressive results on the CNSRC 2022 challenge:

  • Fixed Track: The best performing system achieved a minimum Detection Cost Function (minDCF) of 0.4195.
  • Open Track: The best performing system achieved a minDCF of 0.3707.

These results represent significant improvements, particularly in the challenging fixed track, highlighting the effectiveness of the dynamic convolution and gradient reversal layer techniques.

Limitations and Future Work

  • Gradient Reversal Layer Performance: While the GRL was used to decouple scene information, the authors note that its effect did not fully meet their expectations, and the EER sometimes worsened after fine-tuning. They suggest that exploring more efficient feature decoupling methods is a key direction for future work.
  • Understanding EER Degradation: The paper mentions a need to delve deeper into why the Equal Error Rate (EER) might worsen after fine-tuning, indicating an area for further investigation.
  • Further Research on Feature Decoupling: The authors explicitly state their intention to continue researching more effective ways to decouple features and reduce scene interference.

Summary updated

BibTeX
@misc{zhang2022cross,
  author = {Jialin Zhang and Qinghua Ren and Youcai Qin and Zikai Wan and Qirong Mao},
  title = {{Cross-Scene Speaker Verification Based on Dynamic Convolution for the CNSRC 2022 Challenge}},
  year = {2022},
  doi = {10.21437/odyssey.2022-51},
}
Figure