TL;DR
This paper introduces the voiceHome corpus, a novel French dataset designed for distant-microphone speech processing in realistic home environments. It addresses the critical need for diverse, ecologically valid data to train robust speech technologies that can function effectively amidst the complex reverberation and noise found in everyday homes.
What Problem Does This Paper Address?
Processing speech captured by microphones far from the speaker (distant-microphone speech processing) is a significant challenge. This is especially true in domestic environments, which are characterized by high reverberation, non-stationary and complex noise (e.g., TV, other people talking, appliances), and variable speaker distances. Existing corpora, while valuable, often lack the ecological validity, diversity of environments, or public availability needed to develop truly robust systems for applications like smart home automation and voice control. The authors aim to fill this gap by providing a comprehensive, publicly available French corpus recorded in real homes.
Methods
The voiceHome corpus was created using an 8-microphone array mounted on a cube, capturing audio at 16-bit, 16 kHz. The recording protocol involved native French speakers reading prompts designed for home automation and multimedia applications. These prompts included questions, wishes, and commands, often preceded by a wake-up word ("OK Vesta").
Key methodological contributions include:
- Realistic Recording Environment: Data was collected in a furnished experimental facility mimicking a real home, with microphones fixed but speakers positioned at various points.
- Diverse Conditions: Recordings were made under various noise conditions (e.g., vacuum cleaner, television, dishes washing) and in different rooms.
- Room Impulse Responses (RIRs): RIRs were meticulously recorded in 12 different rooms across 3 real homes, using a chirp signal and a loudspeaker at multiple positions. This allows for the simulation of various acoustic environments.
- Noise Recordings: Extensive recordings of everyday noises were also collected in these rooms.
- Baseline Tools: The authors provide baseline software tools for multichannel source localization (SRP-PHAT implementation) and speech enhancement (using the FASST toolbox with multichannel NMF).
- Speech Recognition Baseline: A speech recognition experiment was conducted using a DNN-HMM acoustic model trained on the ESTER corpus and tested on a subset of voiceHome.
Datasets
The primary contribution is the voiceHome corpus, which includes:
- Audio Data: Clean speech, reverberated speech, and noisy (reverberated) speech from 3 speakers.
- Room Impulse Responses (RIRs): 188 RIRs from 12 rooms in 3 real homes.
- Noise Recordings: 120 minutes of various noise signals.
- Annotations and Transcriptions: Detailed metadata including speaker positions, room geometry, noise conditions, and accurate transcriptions of utterances.
The corpus is designed to be publicly available at no cost, differentiating it from some other similar datasets. For baseline experiments, the ESTER corpus was used for training the speech recognition model.
Key Insights and Ideas
- Ecological Validity is Crucial: The paper strongly emphasizes that real-world speech data, with its inherent variability in noise, reverberation, and speaker behavior, is essential for developing truly robust speech processing systems.
- RIRs as a Powerful Simulation Tool: The availability of RIRs and noise recordings allows researchers to synthesize realistic speech data for training, overcoming the limitations of collecting vast amounts of real data. This is particularly important for machine learning approaches that benefit from large, diverse training sets.
- Complexity of Domestic Noise: The corpus highlights the significant challenge posed by domestic noise, which is often non-stationary, multi-source, and can have diffuse spatial distributions, making localization and separation difficult.
- Trade-offs in Localization: Baseline results show that while speaker localization can achieve high recall with a few estimated directions, accurately identifying noise sources is more challenging and requires more complex processing.
Key Results
- Speaker Localization: The baseline SRP-PHAT algorithm achieved 98% recall for speaker localization with only two estimated directions and a short analysis duration (wake-up word). The average error was low (0.96° azimuth, 2.16° elevation).
- Speech Enhancement: For the "noiseless" condition, the average input SNR was 20.6 dB, and speech enhancement by source separation improved the output SNR to 13.1 dB on average across different noise conditions (ranging from 12.4 dB to 15.3 dB).
- Speech Recognition: Baseline speech recognition on a subset of the corpus yielded Word Error Rates (WER) of 67.93% for home automation prompts and 71.30% for multimedia prompts, indicating the difficulty of the task with current methods.
Limitations and Future Work
- Limited Speaker Diversity: At the time of writing, the corpus only included 3 speakers (2 male, 1 female). The authors plan to increase the number of speakers.
- Constrained Scenarios: While aiming for ecological validity, the initial recordings were still somewhat controlled. Future work will involve collecting speech in more natural dialog scenarios and truly ecological situations, potentially using a Wizard-of-Oz setup.
- Data Augmentation Potential: The authors explicitly mention that the RIRs and noise recordings are key for generating synthetic data to overcome the limited quantity of real speech.
- Future Baselines: The paper states that improved speech recognition baselines, incorporating signal enhancement, model adaptation, and multi-condition training, will be made available in the future.
Summary updated