Abstract
In real-world smart speaker deployments, training data is often harvested automatically, leading to significant label noise when devices are shared among multiple users. This paper introduces a fault-tolerant speaker verification framework designed to mitigate the impact of mislabeled data. By focusing on reducing the model's "condition number" through entropy loss regularization, segment reshuffling, and co-training, the authors achieve a relative error rate reduction of over 35% on a large-scale corpus of 22 million utterances.
Introduction
The rapid proliferation of smart speakers has enabled the collection of massive speech datasets. However, unlike human-curated datasets like NIST SRE, these "in-the-wild" datasets often rely on device IDs for labeling. Because multiple individuals frequently share a single device, the resulting training data is often mislabeled. Previous research indicates that even a 10% mislabeling rate can degrade the Equal Error Rate (EER) of x-vector systems from ~2.7% to nearly 13%. This paper proposes a robust training methodology to maintain high verification accuracy despite such label noise.
Methodology
The authors propose improving "fault tolerance" by minimizing the system's Condition Number—a metric representing the sensitivity of the model's output to small input perturbations, such as incorrect labels. Three key strategies are employed:
Entropy Loss Regularization
Standard cross-entropy loss assumes "hard" ground truth labels. To handle uncertainty, the authors introduce a regularized loss function that "softens" the penalty for mislabeled samples. This prevents the model from making drastic, potentially incorrect weight updates during backpropagation when encountering noisy data.
Regularization by Segments Reshuffling
A novel "lemonade from lemons" approach is used for low-confidence utterances. Instead of discarding them, these utterances are partitioned into segments and appended to high-confidence utterances from the same speaker. This process effectively treats the mislabeled voice as realistic background noise, helping the model learn to distinguish the target speaker from common interference.
Co-Training
The system utilizes two separate classifiers that view the data through distinct feature spaces. By sharing high-confidence labels between the two models, each classifier acts as a regularizer for the other, enhancing the overall robustness of the verification system.
Experimental Results
The proposed framework was evaluated on the NIST SRE (04-10) dataset with synthetic noise and the Tmall Genie Corpus, which contains 22 million real-world utterances.
As shown in the results, the regularization techniques provide substantial gains. On the NIST SRE dataset with 10% mislabeled data, the EER dropped from 12.81% to 6.95%. On the massive Tmall Genie corpus, the proposed system reduced the EER from a baseline of 9.80% to 6.33%, representing a relative improvement of over 35%. These results confirm that combining model-level regularization with data-level reshuffling is highly effective for large-scale, noisy datasets.
Conclusion
This work demonstrates that mathematical regularization of the condition number is a viable alternative to exhaustive data cleaning in large-scale speaker verification. By treating mislabeled data as a resource for augmentation rather than a liability, the proposed system achieves high fault tolerance. Future research will explore more sophisticated labeling quality measures to further refine the training process.
Summary updated