Discussion on domain generalization in the cross-device speaker verification system

Wei-Ting Lin, Yu-Jia Zhang, Chia-Ping Chen, Chung-Li Lu, Bo-Cheng ChanROCLING 2021

Abstract

Speaker verification systems often suffer from performance degradation when users switch recording devices, a phenomenon known as the "cross-device gap." This study explores Domain Generalization (DG) techniques to mitigate device-specific acoustic biases. By employing the ECAPA-TDNN architecture as a backbone and evaluating strategies like DANN, CDANN, and Deep CORAL, the research demonstrates that statistical alignment via Deep CORAL can significantly improve robustness, reducing the average Equal Error Rate (EER) from 18.39% to 8.84% in cross-device scenarios.

Introduction

The "cross-device gap" represents a significant hurdle for real-world speaker verification applications, such as voice-based banking or virtual assistants. Ideally, these systems should recognize a user's unique vocal characteristics regardless of the microphone used. However, different hardware—ranging from high-fidelity studio microphones to standard smartphones and landlines—introduces specific acoustic biases. Standard deep learning models tend to overfit to these device-specific traits, leading to a "domain shift" and subsequent failure when encountering unfamiliar recording environments. This paper investigates the use of Domain Generalization (DG) to extract device-invariant features, effectively bridging the performance gap across different hardware.

Methodology

The researchers implemented a state-of-the-art speaker verification system and evaluated several DG strategies to enhance its robustness.

Backbone Architecture: ECAPA-TDNN

The primary architecture used is the Emphasized Channel Attention, Propagation, and Aggregation Time Delay Neural Network (ECAPA-TDNN). This model is highly effective at extracting "speaker embeddings"—compact mathematical representations of a person's voice—by focusing on channel-wise dependencies and multi-scale feature aggregation.

Architecture of the ECAPA-TDNN backbone used for speaker embedding extraction.

Domain Generalization Strategies

Four distinct approaches were compared to handle the domain shift:

  1. Baseline: Standard training without specific domain-aware loss functions.
  2. DANN (Domain-Adversarial Neural Networks): This method utilizes a Gradient Reversal Layer (GRL) to train a sub-network to identify the recording device. The main network is simultaneously trained to "trick" this classifier, forcing it to learn features that are agnostic to the device.
  3. CDANN (Conditional DANN): An extension of DANN that incorporates speaker labels into the adversarial process to ensure that while device information is suppressed, critical speaker-identifying information is preserved.
  4. Deep CORAL (Correlation Alignment): Unlike adversarial methods, Deep CORAL minimizes the distance between the second-order statistics (covariance) of different device domains, aligning the data distributions across all hardware types.
The Domain-Adversarial Neural Network (DANN) framework featuring a Gradient Reversal Layer (GRL) to encourage device-invariant feature extraction.

Experimental Setup

The study leveraged three datasets for training and evaluation:

  • VoxCeleb2: A large-scale dataset used for initial pre-training of the ECAPA-TDNN backbone.
  • CHT-TDSV: Provided by Chunghwa Telecom Laboratories, this dataset includes 32 speakers recorded across three devices (Mic, Mobile, PSTN), used for fine-tuning with DG algorithms.
  • NSYSU-TDSV: A self-collected evaluation set featuring 12 speakers in three scenarios: studio microphone ("micro"), smartphone ("mobile"), and standard office environment ("office").

Results and Discussion

The evaluation focused on 10 different cross-device combinations. The results highlighted several key insights:

  • Deep CORAL Superiority: Deep CORAL emerged as the most effective strategy, achieving a consistent 8.84% EER overall. In the challenging "micro to mobile" scenario, it reduced the error from a baseline of 13.25% to just 5.30%. This suggests that statistical distribution alignment is more stable for speaker verification than adversarial "confusion" techniques.
  • Adversarial Performance: DANN also showed strong results (8.84% EER), though it exhibited instability in certain device pairs. CDANN performed poorly (13.19% EER), likely due to the increased complexity of the training manifold when conditioning on speaker IDs.
  • Importance of Fine-tuning: The significant performance gains underscore that pre-training on massive datasets like VoxCeleb2 is insufficient for cross-device robustness; targeted fine-tuning with domain-aware losses is essential for real-world deployment.

Conclusion

This research demonstrates that Domain Generalization, particularly through Deep CORAL, can effectively mitigate the acoustic biases introduced by different recording devices. Future work will explore Self-Supervised Learning (SSL) and advanced masking techniques like SAND-mask to further enhance generalization without the need for extensive labeled cross-device data.

Summary updated

BibTeX
@inproceedings{lin2021discussion,
  author = {Wei-Ting Lin and Yu-Jia Zhang and Chia-Ping Chen and Chung-Li Lu and Bo-Cheng Chan},
  title = {{Discussion on domain generalization in the cross-device speaker verification system}},
  booktitle = {ROCLING},
  year = {2021},
}
Figure