The I2R’s ASR System for the VOiCES from a Distance Challenge 2019

Tze Yuang Chong, Kye Min Tan, Kah Kuan Teh, Chang Huai You, Hanwu Sun, Huy Dat TranInterspeech 2019

TL;DR

This paper presents an Automatic Speech Recognition (ASR) system designed for the VOiCES from a Distance Challenge 2019, focusing on recognizing speech in noisy and reverberant conditions with limited training data. The system achieves a competitive average Word Error Rate (WER) of 27.04% by employing multi-style training, de-reverberation techniques, an LSTM language model for rescoring, and system fusion.

What Problem Does This Paper Address?

Real-world speech recognition systems often struggle with the "cocktail party effect" – understanding speech in environments filled with background noise and echoes (reverberation). This is particularly challenging when the training data doesn't accurately reflect these real-world conditions, leading to a significant performance drop. The VOiCES from a Distance Challenge 2019 specifically targets this problem by providing a dataset with limited clean training data but testing on noisy and reverberant speech. The authors aim to build a robust ASR system that can effectively handle this mismatch and data scarcity.

Methods

The I2R system tackles the challenge through a multi-pronged approach:

  • Multi-Style Training: To bridge the gap between clean training data and noisy/reverberant testing data, the authors artificially augment the training set. This involves convolving clean speech with simulated Room Impulse Responses (RIRs) and adding various types of noise (foreground, babble, music). This process is repeated with different noise sources and even speed perturbations to create a more diverse training set.
  • WPE De-reverberation: Before feeding audio into the acoustic models, the Weighted Prediction Error (WPE) algorithm is applied. This technique uses linear prediction to reduce the impact of late reverberation, a major contributor to speech recognition errors in echoey environments.
  • LSTM Language Model Rescoring: Traditional n-gram language models struggle with limited training data. To overcome this, the authors use a Long Short-Term Memory (LSTM) recurrent neural network-based language model. This model is used to rescore the output lattices from the acoustic models, effectively correcting errors and improving the overall recognition accuracy.
  • System Fusion: To further boost performance, the system combines the outputs of eight different acoustic models, each with a distinct neural network architecture (combinations of CNN, TDNN, and LSTM). This fusion process, using weighted combinations of lattices, leverages the strengths of each individual model.

Datasets

  • Training Data: 80.29 hours of clean speech from the LibriSpeech corpus. This data was augmented as described in the "Methods" section.
  • Development Data: Speech from the LibriSpeech corpus that was re-recorded under various reverberant and noise conditions. This data was split into subsets based on room conditions ("rm1", "rm2") and noise types ("babb", "musi", "none", "tele"). The "none" condition specifically refers to reverberation without added noise.

Key Insights and Ideas

  • Data Augmentation is Crucial: The paper strongly emphasizes that artificially adding diverse noise and reverberation to the training data significantly improves robustness. The results show that increasing the number of noise sources and duplication consistently reduces WER.
  • LSTM Language Models Excel with Sparse Data: The LSTM language model's ability to rescore lattices provided a substantial WER reduction, outperforming data augmentation and de-reverberation individually. This highlights its effectiveness in mitigating the impact of limited training text data, a common issue in specialized ASR tasks.
  • De-reverberation Provides Consistent Gains: While not as impactful as LSTM rescoring, the WPE algorithm consistently reduced WER across all tested conditions, demonstrating its value as a front-end processing step.
  • System Fusion is a Powerful Ensemble Technique: Combining multiple acoustic models through lattice fusion consistently yielded better results than any single subsystem, underscoring the benefit of ensemble methods in ASR.

Key Results

The paper reports an average Word Error Rate (WER) of 27.04% on the development set.

  • Data augmentation alone reduced WER significantly. For example, one configuration showed a reduction from 50.05% to 41.83% on a specific subset.
  • WPE de-reverberation reduced the WER by approximately 4.5% on average across different conditions.
  • LSTM language model rescoring provided the most substantial improvement, reducing WER by an average of 17.45%.
  • System fusion further improved performance, with the best fused system achieving lower WERs than individual subsystems.

Limitations and Future Work

  • Limited Training Data: The authors acknowledge the challenge of having only 80 hours of clean training data, which impacts both acoustic and language model training. While their strategies mitigate this, more data would likely lead to further improvements.
  • Arbitrary Fusion Weights: The weights for system fusion were determined arbitrarily based on single-system performance rather than fine-tuned on the development set. The authors suggest that fine-tuning these weights could potentially yield even better results.
  • Future Work: The paper doesn't explicitly detail future work, but the success of multi-style training, WPE, LSTM rescoring, and system fusion suggests that further exploration and refinement of these techniques would be beneficial for future ASR systems operating in challenging acoustic environments.

Summary updated

BibTeX
@inproceedings{chong2019i,
  author = {Tze Yuang Chong and Kye Min Tan and Kah Kuan Teh and Chang Huai You and Hanwu Sun and Huy Dat Tran},
  title = {{The I2R’s ASR System for the VOiCES from a Distance Challenge 2019}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure