Abstract
Speaker verification systems often degrade significantly when there is a language mismatch between training and testing data. Being able to improve cross-lingual speaker verification systems using unlabeled data can greatly increase the robustness of the system and reduce human labeling costs. In this study, we introduce an unsupervised Adversarial Discriminative Domain Adaptation (ADDA) method to effectively learn an asymmetric mapping that adapts the target domain encoder to the source domain, where the target domain and source domain are speech data from different languages. ADDA, together with a popular Domain Adversarial Training (DAT) approach, are evaluated on a cross-lingual speaker verification task: the training data is in English from NIST SRE04-08, Mixer 6 and Switchboard, and the test data is in Chinese from AISHELL-I. We show that with the ADDA adaptation, the Equal Error Rate (EER) of the x-vector system decreases from 9.331% to 7.645%, a relative 18.07% reduction of EER, and a 6.32% reduction from DAT as well. Further data analysis of ADDA adapted speaker embeddings shows that the learned speaker embeddings can perform well on speaker classification for the target domain data and are less dependent on the shift in language.
Introduction
Speaker Verification (SV) systems aim to confirm a person's identity from their voice. While modern systems achieve high accuracy, they are sensitive to "domain shift," particularly language mismatch. A system trained on English speech often struggles with Chinese speech because it may confuse linguistic variations with speaker-specific traits. Traditional solutions involve large-scale labeled data collection in the target language, which is costly and slow. This paper explores using unlabeled target-language data to bridge this gap through unsupervised domain adaptation.
Methodology
The study utilizes the x-vector architecture, a state-of-the-art system using Time-Delay Deep Neural Networks (TDNN) for speaker embedding extraction. Two adversarial adaptation strategies are compared:
- Domain Adversarial Training (DAT): A common method where a single shared encoder tries to learn features that represent the speaker while a discriminator tries to guess the language. A gradient reversal layer forces the encoder to produce features that make the language indistinguishable.
- Adversarial Discriminative Domain Adaptation (ADDA): The proposed method uses separate encoders for source and target domains.
In ADDA, the source encoder is first pre-trained on English data and then frozen. A target encoder is then trained adversarially to map Chinese speech features into the same embedding space as the source encoder. This asymmetric mapping allows the model to account for the unique acoustic properties of the target language more flexibly than the symmetric DAT approach.
Experimental Setup
- Source Domain (English): A massive collection including NIST SRE04-08, Mixer 6, and Switchboard datasets used to train the initial x-vector model.
- Target Domain (Chinese): The AISHELL-1 corpus. The authors used the training set (120,098 utterances) as unlabeled data for the domain adaptation process and evaluated performance on the test set (7,176 utterances from 20 speakers).
Results and Discussion
The performance was evaluated using the Equal Error Rate (EER)—the point where false acceptances and false rejections are equal. The baseline x-vector system without adaptation yielded an EER of 9.331%. Applying DAT reduced this to 8.741%. The proposed ADDA method achieved the best performance with an EER of 7.645%, representing an 18.07% relative reduction over the baseline and a 12.54% improvement over the DAT method.
The results indicate that ADDA's asymmetric mapping is superior for cross-lingual adaptation. By freezing the source encoder, the system preserves its discriminative power while the target encoder learns to align with the established speaker space. Data analysis via t-SNE visualizations confirmed that ADDA leads to tighter speaker clusters that are less influenced by language-specific noise.
Conclusion
This work demonstrates that unsupervised ADDA effectively mitigates performance degradation in cross-lingual speaker verification. By mapping target language features into a pre-trained source space, the system achieves significant error reduction without requiring labeled target data. Future research will investigate the impact of specific phonetic content and integrate phoneme decoders to further isolate linguistic interference from speaker identity.
Summary updated