The CHiME corpus: a resource and a challenge for computational hearing in multisource environments

Heidi Christensen, Jon Barker, Ning Ma, Phil D. GreenInterspeech 2010

TL;DR

This paper introduces the CHIME corpus, a novel dataset designed to push the boundaries of noise-robust speech recognition in realistic, everyday environments. By carefully mixing clean speech with naturalistic, multi-source background noise at controlled signal-to-noise ratios (SNRs), CHIME offers a challenging benchmark for developing ASR systems that can perform well outside of controlled laboratory settings.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems have made significant strides, but their widespread adoption in everyday life is hampered by a lack of robustness to real-world acoustic conditions. Current ASR research often relies on simplified noise models or close-talking microphones, which don't reflect the challenges of human-computer interaction in environments like homes or public spaces. These environments are characterized by multiple, simultaneous, and often unpredictable sound sources, leading to complex acoustic clutter and low SNRs. The CHIME project aims to bridge this gap by creating a dataset that accurately simulates these multisource environments, enabling the development of ASR systems that can truly understand speech in the wild.

Methods

The CHIME corpus was constructed by combining existing clean speech data (from the Grid corpus) with meticulously recorded background noise. The core methodology involves:

  • Naturalistic Background Recording: Binaural recordings were made using a head and torso simulator (HATS) in a domestic setting (lounge and kitchen). This captured the complex acoustic characteristics of these environments, including a variety of stationary and transient noise sources.
  • Binaural Room Impulse Responses (BRIRs): A comprehensive set of BRIRs was collected for different spatial locations within the recording environments. These BRIRs capture how sound propagates and reflects in these spaces.
  • Controlled Mixing: Clean speech utterances from the Grid corpus were artificially spatialized using the collected BRIRs and then mixed with the background recordings. This process allowed for precise control over the target speech SNR, talker location, and other acoustic parameters, while maintaining a naturalistic sound.
  • SNR Definition: A generalized SNR definition was used for binaural signals, considering the energy in both left and right channels.

Datasets

  • Grid Corpus: Used as the source for clean target speech utterances.
  • CHIME Corpus: The newly introduced benchmark dataset. It comprises:
  • Approximately 10 hours of background recordings (for development and training).
  • A set of binaural room impulse responses (BRIRs) from 14 locations in the lounge and 18 in the kitchen.
  • Approximately 30 hours of mixed data, where Grid utterances are convolved with BRIRs and mixed with background recordings at controlled SNRs ranging from -6 dB to 18 dB.

Key Insights and Ideas

  • The Importance of Acoustic Clutter: The paper emphasizes that real-world environments are not just noisy but acoustically "cluttered" with multiple, diverse sound sources. This complexity is a significant challenge for current ASR systems.
  • Controlled Realism: The CHIME corpus strikes a balance between naturalistic recording and controlled experimentation. By using BRIRs and controlled mixing, researchers can systematically evaluate ASR performance across a range of realistic SNRs without sacrificing the ability to compare results directly.
  • Binaural Advantage: The choice of a binaural recording setup is justified by its sufficiency for robust speech processing in humans, offering a practical approach to capturing spatial cues crucial for source separation.
  • Limitations of Existing Datasets: The authors highlight that datasets like Aurora, while useful, often use artificial noise addition that doesn't reflect natural SNR variations, making them less representative of real-world scenarios.

Key Results

The paper presents baseline recognition results using standard ASR techniques (MFCC-based HMMs) on the CHIME corpus. These results demonstrate the challenge posed by the dataset:

  • An MFCC-based system achieved approximately 60% keyword accuracy at 0 dB SNR.
  • Adding cepstral mean normalization (CMN) improved performance to around 70% at 0 dB SNR.
  • A multi-condition training approach, where the training data was also mixed with CHIME noise, yielded the best results, achieving over 80% keyword accuracy at 0 dB SNR.
  • Performance degrades significantly at lower SNRs, illustrating the need for more robust algorithms.

Limitations and Future Work

  • Speech Production Modeling: The current mixing technique does not model how speech production might change in noisy environments (e.g., talkers raising their voices). This is acknowledged as a limitation.
  • Future Challenge: The CHIME corpus is intended to be the basis for a future research challenge focused on binaural speech separation and recognition in multisource environments, expected to be announced in late 2010.
  • Data Distribution: The corpus is planned to be made freely available to the public.

Summary updated

BibTeX
@inproceedings{christensen2010chime,
  author = {Heidi Christensen and Jon Barker and Ning Ma and Phil D. Green},
  title = {{The CHiME corpus: a resource and a challenge for computational hearing in multisource environments}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure