Abstract
This paper investigates a novel data augmentation approach to train deep neural networks (DNNs) used for speaker embedding. By generating new speakers via voice conversion (speed perturbation), we expand the coverage of speakers in the embedding space, improving the model's discriminative power. We also utilize bandwidth extension (BWE) to bridge the gap between narrowband and wideband audio. Experiments on the Speakers in the Wild (SITW) and NIST SRE datasets demonstrate significant reductions in detection costs, with speaker augmentation alone providing a 17.9% improvement.
Introduction
Deep learning models for speaker recognition, such as the x-vector architecture, rely on learning highly discriminative features to separate unique voices. To achieve high performance, these models require training on a massive variety of speakers. However, collecting and labeling data from tens of thousands of unique individuals is expensive and difficult. While standard "audio augmentation" (adding noise or reverb) helps models handle environmental variability, it does not increase the number of speaker identities the model learns to distinguish. This paper addresses this "identity bottleneck" by introducing speaker augmentation and bandwidth extension to artificially expand the training set.
Methods
The authors propose two primary strategies to enhance speaker embedding training:
- Speaker Augmentation via Voice Conversion: Unlike traditional augmentation that keeps the speaker label the same, this method uses speed perturbation (changing pitch and tempo via resampling) to create a "new" voice. These perturbed versions are assigned unique speaker IDs. For example, by creating two copies (at 0.9x and 1.1x speed) of $N$ original speakers, the model is trained to classify $3N$ distinct speaker identities.
- DNN-based Bandwidth Extension (BWE): To utilize both narrowband (8kHz) and wideband (16kHz) data, a pre-processing DNN takes 23 log mel-filterbank coefficients (MFBs) from narrowband speech and estimates the missing 9 high-frequency MFBs. This produces a full 32-MFB wideband-style feature, allowing a single wideband x-vector extractor to be trained on a much larger, mixed-bandwidth pool of speakers.
Key Insights and Ideas
- Identity vs. Variability: Traditional augmentation targets within-class variability (robustness to noise). Speaker augmentation targets between-class discriminability (telling similar people apart).
- Virtual Speakers are Effective: A surprising finding is that "virtual" speakers created by simple signal processing are as effective for training as adding real speakers from a different corpus.
- The Power of Scale: This approach brings the "massive identity" philosophy common in face recognition to the audio domain without the prohibitive cost of data collection.
- Complementary Techniques: Bandwidth extension allows the model to access more data sources, while speaker augmentation multiplies the identities within those sources.
Key Results
The effectiveness of these methods was validated on the Speakers in the Wild (SITW) and NIST SRE 2018 benchmarks:
- SITW Performance: Speaker augmentation reduced the minimum Detection Cost Function (mDCF) from 0.330 to 0.271—a 17.9% improvement. The Equal Error Rate (EER) also dropped from 3.28% to 2.73%.
- NIST SRE 2018: The combination of mixed-bandwidth training, BWE, and speaker augmentation yielded the best results, achieving an mDCF of 0.151 and an EER of 2.18%, significantly outperforming the wideband baseline.
Limitations and Future Work
While effective, the authors used basic speed perturbation for speaker augmentation. More sophisticated voice conversion or synthesis techniques could potentially create even more diverse virtual speakers. Additionally, while BWE helps bridge the frequency gap, other domain differences (like specific acoustic characteristics of telephone handsets) still pose challenges for future research.
Summary updated