Does Speech Enhancement of Publicly Available Data Help Build Robust Speech Recognition Systems? (Student Abstract)

Bhavya Ghai, Buvana Ramanan, Klaus MuellerAAAI 2020 · DOI

Introduction

Training robust Automatic Speech Recognition (ASR) systems typically requires massive amounts of high-quality, private data. This creates a significant "data gap" for researchers and startups who must rely on publicly available audio, which is often contaminated with background noise, music, or traffic. This paper investigates whether automated speech enhancement can bridge this gap by cleaning noisy public data to a level sufficient for building competitive ASR models.

Methodology

The researchers proposed a two-stage strategy that combines a "front-end" enhancement model with "back-end" robust training.

The proposed ASR training pipeline, showing how noisy public data is processed through speech enhancement before being used alongside original noisy data for ASR training.
  1. Front-end Enhancement (SEGAN): The system utilizes a Speech Enhancement Generative Adversarial Network (SEGAN). Unlike traditional methods that operate on frequency representations, SEGAN works directly on raw audio waveforms using a Convolutional Neural Network (CNN) architecture, making it faster and more flexible.
  2. Back-end Multi-condition Training: Instead of training solely on cleaned audio, the ASR model (DeepSpeech) is trained on a mixture of the original noisy audio and the SEGAN-enhanced versions. This multi-condition training encourages the model to remain invariant across different acoustic environments.

Experimental Setup

To test this approach, the authors curated a custom dataset to simulate real-world conditions:

  • Clean Base: LibriSpeech (~460 hours of clean audiobooks).
  • Noise Sources: Clean audio was "contaminated" with diverse noises from Urbansound, ESC50, and YouTube (e.g., traffic, restaurants, and various music genres).
  • Final Dataset: Approximately 205 hours of noisy/clean pairs were used for training and evaluation.

Results and Discussion

The effectiveness of the approach was measured using Word Error Rate (WER). The results demonstrate that the hybrid "Noisy + Enhanced" training strategy significantly outperforms training on noisy data alone.

Word Error Rate (WER) comparison across different training conditions on both clean and noisy test sets.

Key findings include:

  • Significant Performance Gains: Training on "Noisy + Enhanced" data achieved a 32% WER on noisy test data, representing a ~9.5% absolute reduction compared to the 42% WER of the noisy-only model.
  • Parity with Ground Truth: Interestingly, the SEGAN-enhanced approach (32% WER) performed slightly better than the "ideal" scenario of training on Noisy + Ground Truth Clean data (33% WER) when tested on noisy signals, proving the robustness of the enhancement artifacts.
  • Generalization: On clean test data, the hybrid model achieved 27% WER, a marked improvement over the 36% WER of the model trained only on noisy data.

Conclusion

This research demonstrates that high-quality ASR systems do not strictly require private, pristine datasets. By leveraging speech enhancement on noisy public data and employing multi-condition training, researchers can build robust systems that rival those trained on "ideal" ground-truth data. Future work will explore more advanced enhancement architectures and the potential for joint optimization of the enhancement and recognition blocks.

Summary updated

BibTeX
@inproceedings{ghai2020does,
  author = {Bhavya Ghai and Buvana Ramanan and Klaus Mueller},
  title = {{Does Speech Enhancement of Publicly Available Data Help Build Robust Speech Recognition Systems? (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7168},
}
Figure