Introduction
Training robust Automatic Speech Recognition (ASR) systems typically requires massive amounts of high-quality, private data. This creates a significant "data gap" for researchers and startups who must rely on publicly available audio, which is often contaminated with background noise, music, or traffic. This paper investigates whether automated speech enhancement can bridge this gap by cleaning noisy public data to a level sufficient for building competitive ASR models.
Methodology
The researchers proposed a two-stage strategy that combines a "front-end" enhancement model with "back-end" robust training.
- Front-end Enhancement (SEGAN): The system utilizes a Speech Enhancement Generative Adversarial Network (SEGAN). Unlike traditional methods that operate on frequency representations, SEGAN works directly on raw audio waveforms using a Convolutional Neural Network (CNN) architecture, making it faster and more flexible.
- Back-end Multi-condition Training: Instead of training solely on cleaned audio, the ASR model (DeepSpeech) is trained on a mixture of the original noisy audio and the SEGAN-enhanced versions. This multi-condition training encourages the model to remain invariant across different acoustic environments.
Experimental Setup
To test this approach, the authors curated a custom dataset to simulate real-world conditions:
- Clean Base: LibriSpeech (~460 hours of clean audiobooks).
- Noise Sources: Clean audio was "contaminated" with diverse noises from Urbansound, ESC50, and YouTube (e.g., traffic, restaurants, and various music genres).
- Final Dataset: Approximately 205 hours of noisy/clean pairs were used for training and evaluation.
Results and Discussion
The effectiveness of the approach was measured using Word Error Rate (WER). The results demonstrate that the hybrid "Noisy + Enhanced" training strategy significantly outperforms training on noisy data alone.
Key findings include:
- Significant Performance Gains: Training on "Noisy + Enhanced" data achieved a 32% WER on noisy test data, representing a ~9.5% absolute reduction compared to the 42% WER of the noisy-only model.
- Parity with Ground Truth: Interestingly, the SEGAN-enhanced approach (32% WER) performed slightly better than the "ideal" scenario of training on Noisy + Ground Truth Clean data (33% WER) when tested on noisy signals, proving the robustness of the enhancement artifacts.
- Generalization: On clean test data, the hybrid model achieved 27% WER, a marked improvement over the 36% WER of the model trained only on noisy data.
Conclusion
This research demonstrates that high-quality ASR systems do not strictly require private, pristine datasets. By leveraging speech enhancement on noisy public data and employing multi-condition training, researchers can build robust systems that rival those trained on "ideal" ground-truth data. Future work will explore more advanced enhancement architectures and the potential for joint optimization of the enhancement and recognition blocks.
Summary updated