Impulse Response Data Augmentation and Deep Neural Networks for Blind Room Acoustic Parameter Estimation

Nicholas J. BryanICASSP 2020 · DOI

TL;DR

This paper introduces a novel data augmentation technique for acoustic impulse responses (AIRs) that allows for parametric control of reverberation time (T60) and direct-to-reverberant ratio (DRR). By augmenting a small dataset of real AIRs, they train deep neural networks that achieve state-of-the-art performance in blind estimation of these room acoustic parameters, while also being significantly faster than previous methods.

What Problem Does This Paper Address?

Estimating room acoustic parameters like reverberation time (T60) and direct-to-reverberant ratio (DRR) is crucial for applications like audio processing, virtual reality, and speech enhancement. Traditionally, these parameters are measured directly from acoustic impulse responses (AIRs). However, in many real-world scenarios, only the recorded audio (e.g., speech) is available, making direct measurement impossible. This necessitates "blind" estimation methods that infer these parameters directly from the audio signal.

While deep learning (DL) has shown great promise in various audio tasks, its application to blind acoustic parameter estimation has been hampered by the need for large, realistic datasets. Collecting such datasets is expensive and time-consuming. Existing datasets are often small and imbalanced, limiting the effectiveness of deep neural networks. This paper addresses this challenge by proposing a method to artificially expand and balance existing AIR datasets, thereby enabling more robust and accurate blind estimation using DL.

Methods

The core contribution of this paper is a novel Impulse Response Data Augmentation (AIRA) method. This technique allows for the parametric control of T60 and DRR within a recorded AIR.

  • DRR Augmentation: The method decomposes an AIR into its early and late-field components. The early response is further split into a direct path and a residual. By applying a scalar gain to the direct path component and carefully crossfading it back with the residual, the DRR can be adjusted. This process is mathematically formulated to ensure smooth transitions and prevent discontinuities.
  • T60 Augmentation: The late-field response of an AIR is modeled as a frequency-dependent exponentially decaying Gaussian noise with an additive noise floor. The paper utilizes the K-T60 estimator to model this decay. To augment the T60, the estimated decay rate is then modified by multiplying the late-field response with a growing or shrinking exponential. This allows for imposing a desired reverberation time per frequency band. For fullband augmentation, a ratio of desired to estimated decay is computed and applied to subbands.
  • Noise Floor Handling: A critical step in T60 augmentation involves accurately detecting and handling the noise floor to avoid unstable estimations. The authors adapt a procedure that detects the noise floor onset and stitches in a synthesized matching late-field response.

These augmented AIRs are then used to train deep convolutional neural networks (CNNs) for blind T60 and DRR estimation. The authors propose two CNN architectures: one based on the state-of-the-art GT-CNN and a more efficient, straightforward baseline CNN.

Datasets

  • Training/Validation/Testing: The authors created their own dataset by convolving speech recordings from the Device and Produced Speech (DAPS) dataset with their augmented AIRs. Noise from the ACE corpus was also used. This resulted in a large, statistically balanced dataset for training and evaluation.
  • ACE Challenge Data: The paper also references the ACE Challenge dataset, which was used for benchmarking. They used a subset of ACE corpus development set AIRs (Building Lobby and Office 1) for their augmentation process.

Key Insights and Ideas

  • Data Augmentation is Key for DL in Acoustics: The paper strongly suggests that the primary bottleneck for applying deep learning to blind acoustic parameter estimation is the lack of large, balanced datasets. Their AIRA method effectively addresses this by generating a vast number of realistic AIRs from a small initial set.
  • Parametric Control is Powerful: The ability to precisely control T60 and DRR during augmentation is a significant advantage. This allows for the creation of datasets that are not only larger but also specifically tailored to cover a wide range of acoustic conditions, leading to more robust models.
  • Efficiency Matters: While achieving high accuracy is important, the paper also highlights the development of a more efficient CNN architecture that offers a substantial speedup (4-5x faster) without sacrificing performance, making it more practical for real-time applications.
  • Calibration is Crucial: The authors hypothesize that a lack of proper calibration in previous datasets and methods contributed to performance limitations. Their approach of using calibrated ground truth estimators for their dataset is presented as a factor in their success.

Key Results

  • State-of-the-Art Performance: The proposed CNN trained with AIRA significantly outperforms previous single-channel state-of-the-art methods for both T60 and DRR estimation in terms of bias, MSE, and correlation coefficient.
  • Relative Improvements: Using AIRA with the GT-CNN method resulted in relative improvements of +27%, +31%, and +8% for T60 bias, MSE, and correlation, respectively. Their own CNN + AIRA achieved even better improvements. For DRR, their CNN + AIRA showed a 56% improvement in bias, 39% in MSE, and 27% in correlation compared to previous methods.
  • Computational Efficiency: The proposed baseline CNN is 4-5x faster than the GT-CNN method and significantly faster (up to 100x) than other prior methods, achieving a real-time factor (RTF) of 0.0088.

Limitations and Future Work

  • Limitations: The paper acknowledges that while their method creates a statistically balanced dataset, it is still based on augmenting real AIRs. The inherent limitations of the original AIRs (e.g., microphone characteristics) might still influence the augmented data. The authors also note that they did not have access to the ground truth ACE estimators, necessitating calibration of their own estimators.
  • Future Work: The authors suggest that their method can be extended to other acoustic parameters beyond T60 and DRR. They also imply that further exploration of different CNN architectures and training strategies could yield even better results. The potential for real-time applications due to the increased efficiency is also a promising avenue.

Summary updated

BibTeX
@inproceedings{bryan2020impulse,
  author = {Nicholas J. Bryan},
  title = {{Impulse Response Data Augmentation and Deep Neural Networks for Blind Room Acoustic Parameter Estimation}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2020},
  doi = {10.1109/icassp40776.2020.9052970},
}
Figure