Abstract
The COVID-19 pandemic highlighted the limitations of facial recognition systems when users wear face masks. To address this, researchers from WeBank and HKUST developed a speaker verification (SV) system that remains accurate even when speech is muffled by masks. By combining a ResNet-34 feature extractor with a Multi-Head Attention (MHA) mechanism and Additive Margin Softmax (AM-Softmax) loss, the system achieves a remarkably low Equal Error Rate (EER) of 0.076%. This solution provides a secure, touchless, and health-friendly authentication method suitable for mobile deployment.
Introduction
Traditional biometric authentication, particularly facial recognition, often requires users to remove masks, creating potential health risks in public environments. Speaker verification (SV) offers a natural alternative, but physical barriers like masks can distort acoustic patterns. This research focuses on building a robust SV system capable of handling these "complex scenarios" without compromising security or user safety.
Methodology
The system architecture follows a sophisticated three-stage pipeline designed to extract robust speaker identities from potentially distorted audio.
- Speech Processing: Raw audio undergoes Voice Activity Detection (VAD) to remove silence. The remaining signal is converted into log-scaled filter-banks and normalized using Cepstral Mean and Variance Normalization (CMVN).
- Local Acoustic Pattern Extractor: A modified ResNet-34 convolutional neural network extracts frame-level representations, capturing the unique textures of a speaker's voice in short time slices.
- Multi-Head Attention (MHA) Embedding: Instead of simple statistical pooling, the system uses MHA to aggregate frame-level features into a single utterance-level embedding. This allows the model to focus on the most discriminative parts of the speech, effectively ignoring frequencies muffled by a mask.
To enhance reliability, the model is optimized with Additive Margin Softmax (AM-Softmax) loss, which forces a wider mathematical margin between different speakers in the embedding space.
Datasets and Training
The model was trained on a diverse collection of over 40,000 speakers to ensure broad generalization across languages and accents. The training set included:
- VoxCeleb1 & VoxCeleb2: Large-scale celebrity speech datasets.
- Aishell-1 & Aishell-2: Massive open-source Mandarin speech corpora.
- MAGIDATA: Additional robust speech data.
Key Results and Usability
The system demonstrates high performance and practical efficiency for real-world use:
- Accuracy: Achieved an EER of 0.076% in complex testing scenarios.
- Speed: Requires only 10 seconds of speech for registration and 3 seconds for verification.
- Mobile Deployment: The system is optimized for mobile use, as demonstrated by its implementation in a WeChat mini-program for secure login and payments.
Conclusion and Future Work
This "health-friendly" speaker verification system successfully balances security and hygiene. By utilizing attention-based pooling and margin-based learning, it overcomes the acoustic challenges posed by face masks. Future research will focus on further optimizing the model for low-power mobile devices and improving performance in high-noise environments like airports or hospitals.
Summary updated