Introduction
While speaker recognition—the task of identifying who is speaking—has seen dramatic improvements through deep learning, most existing benchmarks rely on relatively clean data like telephone speech or curated YouTube clips (e.g., VoxCeleb). These datasets often fail to represent the "nightmare scenario" for audio systems: natural human interaction in complex physical spaces.
This research addresses the gap in far-field, multi-speaker environments. In typical home settings, microphones are often distant from the speaker, and multiple individuals frequently converse simultaneously. By repurposing the CHiME-5 corpus, originally intended for speech recognition, the authors provide a standardized framework to measure the impact of noise, distance, and overlapping speech on speaker recognition performance.
Methodology
The researchers established a baseline using a state-of-the-art x-vector DNN system. The process involves:
- Feature Extraction: Utilizing 30-dimensional MFCCs to feed a deep neural network that extracts "x-vectors"—fixed-length embeddings representing speaker identity.
- Scoring: Comparing these embeddings via Probabilistic Linear Discriminant Analysis (PLDA) to verify speaker identity.
- Diarization: Employing Agglomerative Hierarchical Clustering (AHC) to separate overlapping audio into individual speaker streams.
The benchmark is structured around four tasks that vary the difficulty of enrollment and testing:
- Core-Core: Clean, annotated speech for both phases.
- Assist-Core: Enrollment uses full audio segments (requiring the system to isolate the speaker), while testing uses clean speech.
- Core-Multi: Clean enrollment paired with multi-speaker test segments.
- Assist-Multi: A "real-world" scenario where both enrollment and testing involve complex, multi-speaker audio.
Dataset and Benchmark Setup
The benchmark is built upon the CHiME-5 corpus, which consists of 20 dinner parties recorded in real homes using both wearable binaural microphones (close-talking) and six Microsoft Kinect devices (far-field). The authors curated a specific subset of 39 speakers from 18 of these parties. To ensure a robust baseline, the x-vector model was pre-trained on the massive VoxCeleb 1 & 2 datasets, encompassing over 7,000 speakers.
Key Results and Insights
The study reveals a significant "Far-Field Gap." The Equal Error Rate (EER) increases nearly 10x when transitioning from wearable microphones (1.2% EER) to room-mounted Kinect arrays (10% to 23% EER).
Key findings include:
- Diarization Impact: In multi-speaker test conditions, applying diarization reduced the EER from 20.1% to 12.2%. While helpful, a substantial performance gap remains compared to single-speaker conditions.
- Enrollment Selectivity: Maintaining high performance in noisy environments requires "highly selective" enrollment—using only the cleanest, non-overlapping audio to define a speaker's profile.
- Device Variability: Performance varies significantly depending on the specific room acoustics and the placement of the far-field device.
Conclusion and Future Directions
The CHiME-5 speaker recognition benchmark highlights the extreme difficulty of far-field, multi-speaker scenarios. Current baselines, which process microphone channels independently, struggle with the degradations inherent in these environments. Future work should explore microphone-array beamforming and advanced speech separation to bridge the performance gap. Furthermore, the marginal gains seen in the "Assist" enrollment condition suggest a need for more sophisticated methods to leverage noisy enrollment data effectively.
Summary updated