IR-GAN: Room Impulse Response Generator for Speech Augmentation.

Anton Ratnarajah, Zhenyu Tang, Dinesh Manocha

Abstract

Far-field speech recognition is significantly challenged by reverberation, necessitating large-scale, realistic Room Impulse Response (RIR) datasets for robust model training. We present IR-GAN, a generative adversarial network tailored to synthesize high-fidelity RIRs. Unlike traditional geometric simulators that often produce overly idealized results, IR-GAN incorporates physical constraints based on acoustic parameters—such as $T_{60}$ and Direct-to-Reverberant Ratio—to ensure the generated audio remains physically plausible. Our evaluation shows that IR-GAN-generated RIRs significantly improve Automatic Speech Recognition (ASR) performance, yielding up to a 14.3% reduction in word error rate (WER) when used in conjunction with traditional simulation techniques.

Introduction

Far-field speech recognition—the ability of a system to understand speech from a distance in reverberant environments—remains a core challenge in audio processing. To train robust models, researchers require vast amounts of reverberant speech data. Traditionally, this data is acquired either through physical recordings of RIRs or via Geometric Acoustic Simulators (GAS). While recordings are accurate, they are difficult to scale. Conversely, GAS methods are fast but often fail to capture the stochastic complexity and "messiness" of real-world acoustic environments.

Methodology

The authors introduce IR-GAN, which adapts the WaveGAN architecture—a 1D adaptation of DCGAN—to the audio domain for RIR synthesis.

Constrained RIR Generation

A primary innovation of IR-GAN is the use of Constrained RIR Generation. Purely data-driven GANs often produce "hallucinated" artifacts that lack physical meaning. To mitigate this, the model extracts four key acoustic metrics from real-world RIRs:

  • Reverberation Time ($T_{60}$)
  • Direct-to-Reverberant Ratio (DRR)
  • Early-Decay Time (EDT)
  • Early-to-Late Index (CTE)

By forcing the GAN to generate RIRs that adhere to the statistical distributions of these parameters, the authors ensure the synthetic outputs are grounded in physical reality.

Comparison of spectrograms between a recorded RIR (top) and a generated RIR (bottom).

As shown in the spectrogram comparison above, IR-GAN successfully captures the decay patterns and spectral characteristics of recorded RIRs. Furthermore, the constrained approach ensures that the generated samples match the statistical profile of the training data.

Histograms showing the distribution of $T_{60}$ values for real training samples versus RIRs generated by IR-GAN.

Experimental Setup

The model was trained and evaluated using several key datasets:

  • BUT ReverbDB: Real-world RIRs used for GAN training and ASR evaluation.
  • AIR (Acoustic Impulse Response) Dataset: An out-of-distribution test set to evaluate generalization.
  • LibriSpeech: Source of clean speech for data augmentation.

The synthetic RIRs were convolved with LibriSpeech samples and mixed with background noise to create a comprehensive training set for ASR models.

Results and Discussion

The effectiveness of IR-GAN was measured by the Word Error Rate (WER) on far-field ASR tasks. Key findings include:

  • Performance Gains: The constrained IR-GAN achieved an 8.95% lower WER than standard GAS on the BUT ReverbDB benchmark.
  • Hybrid Synergy: The best results were achieved by combining IR-GAN and GAS. GAS provides precise geometric reflections, while IR-GAN excels at modeling the complex, stochastic "tail" of reverberation. This hybrid approach yielded a total WER reduction of 14.3%.
  • Generalization: On the unseen AIR dataset, IR-GAN reduced the WER from a baseline of 26.79% to 7.71%, demonstrating its ability to create highly generalizable acoustic profiles.

Conclusion

IR-GAN bridges the gap between deep learning and classical acoustics by incorporating physical constraints into a generative framework. By modeling the complex reverberation tails that geometric simulators often miss, IR-GAN provides a scalable and realistic solution for training far-field ASR systems. Future work will focus on extending this approach to outdoor environments and increasing the diversity of training datasets to include more complex room geometries.

Summary updated

Abstract

We present a Generative Adversarial Network (GAN) based room impulse response generator for generating realistic synthetic room impulse responses. Our proposed generator can create synthetic room impulse responses by parametrically controlling the acoustic features captured in real-world room impulse responses. Our GAN-based room impulse response generator (IR-GAN) is capable of improving far-field automatic speech recognition in environments not known during training. We create far-field speech training set by augmenting our synthesized room impulse responses with clean LibriSpeech dataset. We evaluate the quality of our room impulse responses on the real-world LibriSpeech test set created using real impulse responses from BUT ReverbDB and AIR datasets. Furthermore, we combine our synthetic data with synthetic impulse responses generated using acoustic simulators, and this combination can reduce the word error rate by up to 14.3% in far-field speech recognition benchmarks.
BibTeX
@misc{ratnarajah2020ir,
  author = {Anton Ratnarajah and Zhenyu Tang and Dinesh Manocha},
  title = {{IR-GAN: Room Impulse Response Generator for Speech Augmentation.}},
  year = {2020},
  eprint = {2010.13219},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.13219},
}
Figure