Abstract
Modern speaker verification systems rely on deep neural networks (DNNs) to extract robust speaker embeddings. However, these models often struggle to generalize to unseen speakers. This paper investigates Mixup, a data augmentation strategy that creates synthetic training samples through linear interpolation of existing data pairs. By applying Mixup at the raw audio level, the researchers achieved up to a 13% relative improvement in accuracy across standard benchmarks including NIST SRE and SITW, without the need for external noise libraries.
Introduction
The primary objective in speaker verification (SV) is the extraction of "speaker embeddings" (e.g., x-vectors) that can uniquely represent any human voice, including those not present in the training set. A significant hurdle is the tendency of DNNs to memorize training samples rather than learning generalized vocal characteristics. While traditional data augmentation—such as adding noise or reverberation—can mitigate this, it typically requires large, curated external datasets. This research explores Mixup as a self-contained regularization technique to enhance model generalization using only the existing training data.
Methodology: The Mixup Strategy
Mixup generates "virtual" training examples by blending two random samples $(x_i, y_i)$ and $(x_j, y_j)$ from the training set. The synthetic sample $(\tilde{x}, \tilde{y})$ is defined as:
$$\tilde{x} = \lambda x_i + (1 - \lambda) x_j$$ $$\tilde{y} = \lambda y_i + (1 - \lambda) y_j$$
where $\lambda \in [0, 1]$ is a random weight typically drawn from a Beta distribution. The model is trained to predict these soft, interpolated labels, effectively smoothing the decision boundaries. The authors adapted this strategy to the standard Kaldi x-vector (TDNN) architecture.
The authors evaluated two primary implementations:
- MFCC Feature Mixup: Interpolation of 23-dimensional acoustic features post-extraction.
- Raw Audio Mixup: Interpolation of time-domain waveforms. To handle varying clip lengths, the shorter audio segment is repeated to match the duration of the longer segment before blending.
Experimental Results
The proposed methods were benchmarked on several industry-standard datasets, including NIST SRE 2010, NIST SRE 2016, and Speakers in the Wild (SITW).
NIST SRE 2010 Performance
On the NIST SRE10 evaluation set, raw audio mixup demonstrated a substantial performance boost. The Equal Error Rate (EER) was reduced from 2.50% to 2.18%, representing a 13% relative improvement.
Multilingual Evaluation (NIST SRE 2016)
The effectiveness of Mixup was further validated on the multilingual NIST SRE16 dataset, which includes Cantonese and Tagalog speech. Raw audio mixup consistently outperformed the baseline across both languages.
SITW and VoxCeleb
Even on high-quality 16kHz audio from the SITW dataset, where baselines are already competitive, Mixup provided a 5% relative improvement in EER.
Key Insights
- Raw Audio Superiority: Blending at the waveform level consistently yielded better results than feature-level interpolation. This suggests that raw signal mixing creates a more effective "vicinal distribution" for the network to learn robust features.
- Optimal Distribution: While various Beta distributions were tested, a Uniform distribution ($\alpha=1$) was found to be the most effective and stable, simplifying implementation by removing the need for hyper-parameter tuning.
- Regularization and Accuracy: Mixup acts as a powerful regularizer, preventing the network from becoming over-confident. Notably, it improved internal speaker classification accuracy on the training set by up to 43%, indicating the learning of more discriminative features.
Conclusion
Mixup is a simple yet highly effective strategy for improving the generalization of speaker verification models. By interpolating raw audio signals, researchers can achieve significant gains in EER and DCF across diverse acoustic environments and languages without relying on external data augmentation resources.
Summary updated