TL;DR
This paper introduces Libri-Adapt, a novel 7200-hour speech dataset designed to tackle the challenge of unsupervised domain adaptation in Automatic Speech Recognition (ASR). By recording speech on various embedded microphones, with different accents, and in diverse acoustic environments, Libri-Adapt provides a rich testbed for developing ASR models that are robust to real-world variations.
What Problem Does This Paper Address?
Automatic Speech Recognition (ASR) systems have made incredible strides thanks to deep learning. However, their performance often degrades significantly when faced with data that differs from their training set – a phenomenon known as "domain shift." This shift can arise from numerous real-world factors: the acoustics of the recording environment (e.g., a quiet room vs. a noisy street), variations in speaker accents, or even the type of microphone used. Collecting labeled data for every possible scenario is prohibitively expensive. This paper addresses the critical need for a large-scale, realistic dataset that specifically enables research into unsupervised domain adaptation (UDA) for ASR, allowing models to adapt to new domains without requiring labeled data from those target domains.
Methods
The core contribution of this paper is the creation of the Libri-Adapt dataset. The authors built upon the existing LibriSpeech-clean-100 dataset, which provides clean US English speech. They then re-recorded this corpus using a "replay-and-record" methodology. This involved playing the LibriSpeech audio through a monitor speaker and simultaneously recording it with six different microphones. To introduce domain shifts, they employed:
- Microphone Variability: Six distinct microphones were used, including consumer-grade embedded-scale microphones (like those found in smartphones and microcomputers) and a higher-quality Shure microphone for baseline comparison.
- Accent Variability: Speech was recorded in three English accents: US English, British English, and Indian English. The British and Indian accents were synthesized using Google Cloud's Text-to-Speech API to ensure the underlying text remained the same across accents.
- Acoustic Environment Variability: Four conditions were simulated: Clean, Rain, Wind, and Laughter. Noise samples were sourced from the ESC-50 dataset and overlaid onto the clean recordings at a 20dB SNR.
This systematic combination of factors results in 72 distinct domains (6 microphones x 3 accents x 4 environments). The authors also implemented a network-based synchronization scheme (MQTT) to ensure time-synchronization across simultaneous recordings from multiple microphones.
Datasets
- LibriSpeech-clean-100: The base dataset used for its clean US English speech and transcripts.
- Libri-Adapt: The newly introduced dataset, containing 7200 hours of training data and 5.4 hours of test data per domain, spanning 72 unique domains. This dataset is designed to facilitate UDA research by providing controlled variations in microphones, accents, and acoustic environments.
Key Insights and Ideas
- Microphone Heterogeneity is a Major Domain Shift: The paper strongly emphasizes that variations in embedded microphones, often overlooked, introduce significant domain shifts that can severely impact ASR performance.
- Replay-and-Record is a Practical Approach for Large-Scale UDA Datasets: While acknowledging potential speaker-related artifacts, the authors argue that this method is a cost-effective way to generate large, multi-domain datasets crucial for UDA research.
- UDA is Essential for Robust ASR: The baseline experiments clearly demonstrate that even state-of-the-art models like DeepSpeech2 struggle with domain shifts, highlighting the necessity of UDA techniques for real-world deployment.
- Synergistic Domain Shifts: Libri-Adapt allows for the study of complex scenarios where multiple domain shifts (e.g., a different accent and a different microphone) occur simultaneously, offering a more realistic evaluation of ASR robustness.
Key Results
The paper presents baseline results using the Mozilla DeepSpeech2 ASR model. Key findings include:
- Microphone Shift Impact: A model trained on the high-quality Shure microphone achieved a Word Error Rate (WER) of 10.19% when tested on itself, but this jumped to 24.95% when tested on a Matrix Voice microphone. This clearly illustrates the significant impact of microphone variability.
- Accent Shift Impact: When testing a model trained on US English accents on Indian English accent data (with other conditions kept constant), the WER increased from 11.4% to 29.5%.
- Noise Shift Impact: Domain shifts caused by ambient noise also degrade performance, making it a pertinent problem for domain adaptation research.
- Multi-Domain Shift Impact: Combining shifts, such as testing a model trained on ReSpeaker-US-Clean on PlayStationEye-Indian-Rain data, resulted in a WER increase of nearly 29.8%.
Limitations and Future Work
- Limitations: The authors acknowledge that the "replay-and-record" methodology might introduce minor speaker-related effects due to the transfer function of the replay speaker.
- Future Work: The paper explicitly states their intention to develop unsupervised domain adaptation algorithms to tackle the domain shifts present in the Libri-Adapt dataset. They also plan to release the dataset publicly under a CC BY 4.0 license.
Summary updated