Abstract
Far-field speech recognition is significantly challenged by reverberation, necessitating large-scale, realistic Room Impulse Response (RIR) datasets for robust model training. We present IR-GAN, a generative adversarial network tailored to synthesize high-fidelity RIRs. Unlike traditional geometric simulators that often produce overly idealized results, IR-GAN incorporates physical constraints based on acoustic parameters—such as $T_{60}$ and Direct-to-Reverberant Ratio—to ensure the generated audio remains physically plausible. Our evaluation shows that IR-GAN-generated RIRs significantly improve Automatic Speech Recognition (ASR) performance, yielding up to a 14.3% reduction in word error rate (WER) when used in conjunction with traditional simulation techniques.
Introduction
Far-field speech recognition—the ability of a system to understand speech from a distance in reverberant environments—remains a core challenge in audio processing. To train robust models, researchers require vast amounts of reverberant speech data. Traditionally, this data is acquired either through physical recordings of RIRs or via Geometric Acoustic Simulators (GAS). While recordings are accurate, they are difficult to scale. Conversely, GAS methods are fast but often fail to capture the stochastic complexity and "messiness" of real-world acoustic environments.
Methodology
The authors introduce IR-GAN, which adapts the WaveGAN architecture—a 1D adaptation of DCGAN—to the audio domain for RIR synthesis.
Constrained RIR Generation
A primary innovation of IR-GAN is the use of Constrained RIR Generation. Purely data-driven GANs often produce "hallucinated" artifacts that lack physical meaning. To mitigate this, the model extracts four key acoustic metrics from real-world RIRs:
- Reverberation Time ($T_{60}$)
- Direct-to-Reverberant Ratio (DRR)
- Early-Decay Time (EDT)
- Early-to-Late Index (CTE)
By forcing the GAN to generate RIRs that adhere to the statistical distributions of these parameters, the authors ensure the synthetic outputs are grounded in physical reality.
As shown in the spectrogram comparison above, IR-GAN successfully captures the decay patterns and spectral characteristics of recorded RIRs. Furthermore, the constrained approach ensures that the generated samples match the statistical profile of the training data.
Experimental Setup
The model was trained and evaluated using several key datasets:
- BUT ReverbDB: Real-world RIRs used for GAN training and ASR evaluation.
- AIR (Acoustic Impulse Response) Dataset: An out-of-distribution test set to evaluate generalization.
- LibriSpeech: Source of clean speech for data augmentation.
The synthetic RIRs were convolved with LibriSpeech samples and mixed with background noise to create a comprehensive training set for ASR models.
Results and Discussion
The effectiveness of IR-GAN was measured by the Word Error Rate (WER) on far-field ASR tasks. Key findings include:
- Performance Gains: The constrained IR-GAN achieved an 8.95% lower WER than standard GAS on the BUT ReverbDB benchmark.
- Hybrid Synergy: The best results were achieved by combining IR-GAN and GAS. GAS provides precise geometric reflections, while IR-GAN excels at modeling the complex, stochastic "tail" of reverberation. This hybrid approach yielded a total WER reduction of 14.3%.
- Generalization: On the unseen AIR dataset, IR-GAN reduced the WER from a baseline of 26.79% to 7.71%, demonstrating its ability to create highly generalizable acoustic profiles.
Conclusion
IR-GAN bridges the gap between deep learning and classical acoustics by incorporating physical constraints into a generative framework. By modeling the complex reverberation tails that geometric simulators often miss, IR-GAN provides a scalable and realistic solution for training far-field ASR systems. Future work will focus on extending this approach to outdoor environments and increasing the diversity of training datasets to include more complex room geometries.
Summary updated