Performance estimation of reverberant speech recognition based on reverberant criteria RSR-dn with acoustic parameters

Takahiro Fukumori, Masanori Morise, Takanobu NishiuraInterspeech 2010

TL;DR

This paper introduces a new set of criteria, RSR-Dn, to accurately estimate the performance of speech recognition systems in reverberant environments. By leveraging the ISO3382 acoustic parameter "D value," which quantifies the clarity of sound, the authors demonstrate a significant improvement in prediction accuracy compared to traditional methods relying solely on reverberation time.

What Problem Does This Paper Address?

As speech interfaces become more ubiquitous and are used in diverse environments (e.g., hands-free systems, distant talking), robust speech recognition is crucial. A major challenge in this domain is the performance degradation caused by reverberation – the echoes and reflections of sound in an enclosed space. While reverberation time (T60) is a common metric for characterizing room acoustics, it's insufficient for predicting speech recognition performance. This is because T60 doesn't account for the critical distinction between early reflections (which can sometimes aid perception) and late reflections (which tend to degrade it), nor does it consider the distance between the speaker and the microphone. The authors aim to bridge this gap by proposing a more nuanced metric that can reliably estimate how well a speech recognition system will perform in a given reverberant setting.

Methods

The core of this paper is the development of the RSR-Dn (Reverberant Speech Recognition criteria with Dn). The authors build upon their previous work and the ISO3382 standard for acoustic measurements.

  1. ISO3382 Acoustic Parameters: They focus on the "D value," which measures the clarity of acoustics by quantifying the balance between early and late arriving sound energies. A higher D value indicates better clarity. The D value is calculated using the impulse response of a room, with a specific "border time" (n) defining the cutoff between early and late reflections.
  2. Determining the Optimal Border Time (n): Through experiments, they found that a border time of 20 milliseconds yielded the best correlation between the D value and speech recognition performance. This value was used to calculate Dn for subsequent steps.
  3. Developing RSR-Dn Criteria: The authors then established a relationship between the calculated Dn values and actual speech recognition performance. They employed regression analysis, specifically exploring both linear and quadratic functions, to model this relationship. This led to the definition of RSR-Dn criteria.
  4. RSR-D20L and RSR-D20Q: They specifically define two versions: RSR-D20L (using a linear regression function) and RSR-D20Q (using a quadratic regression function). The quadratic version, RSR-D20Q, proved to be the most effective.
  5. Performance Estimation: The proposed RSR-Dn criteria are then used to estimate speech recognition performance by taking the reverberation time and D value of a test environment as input.

Datasets

  • Training Data: Impulse responses were measured in nine different environments (listed in Table 1(a)), ranging from a soundproof room (T60=100 ms) to a lift station and standard stairs (T60=850 ms). A time-stretched pulse was used for measurement.
  • Speech Samples: An ATR phoneme-balanced set of 216 isolated Japanese words, spoken by 14 speakers, was used.
  • Evaluation Environments: Experiments were conducted in three specific environments (Table 1(b) and (e)) to calculate speech recognition performance and test the RSR-Dn criteria. These included Laboratory (T60=450 ms), Bath room (T60=650 ms), and Lift station (T60=850 ms).

Key Insights and Ideas

  • Reverberation Time (T60) is Insufficient: The paper strongly reinforces that T60 alone is a poor predictor of speech recognition performance, especially in highly reverberant conditions. Figure 1 visually demonstrates this lack of a clear monotonic relationship.
  • Early vs. Late Reflections Matter: The distinction between early and late reflections is critical. Early reflections (within ~12.5 ms) can have a neutral or even slightly positive impact, while later reflections significantly degrade performance. The D value effectively captures this distinction.
  • The "D Value" as a Clarity Metric: The ISO3382 D value emerges as a powerful acoustic parameter for predicting speech recognition performance because it directly quantifies the clarity of the sound field, which is more relevant than just the total decay time.
  • Quadratic Regression for Better Fit: While linear regression showed promise, a quadratic regression model (RSR-D20Q) provided a more accurate estimation of speech recognition performance, suggesting a non-linear relationship between acoustic clarity and system accuracy.
  • Robustness Across Environments: The proposed RSR-D20Q criteria demonstrated good performance even in "open tests," where the criteria were designed in one environment but used to predict performance in a different, unseen environment with similar reverberation characteristics.

Key Results

  • The proposed RSR-D20Q criteria achieved an average estimation error of less than 5% across all tested environments.
  • Correlation coefficients between the estimated and actual speech recognition performance were higher than 0.93 for both RSR-D20L and RSR-D20Q, significantly outperforming conventional methods.
  • Table 2 shows that for environments with T60=400 ms, 600 ms, and 850 ms, the correlation coefficients for RSR-D20Q were consistently high (0.939, 0.963, and 0.972 respectively).
  • Figure 7 illustrates that the average estimation error for RSR-D20Q was substantially lower than conventional methods across different environments and test conditions (closed/open). For example, in the Lift station (T60=850 ms), the error for RSR-D20Q was around 2.41-5.35% compared to 8.80-17.64% for conventional methods.

Limitations and Future Work

  • Limitations: The paper acknowledges that the RSR-Dn criteria are designed based on specific acoustic parameters and may need further refinement for extremely complex acoustic scenarios or different types of speech recognition tasks. The performance estimation is also dependent on the quality of the impulse response measurements.
  • Future Work: The authors suggest exploring the definition of more suitable reverberation criteria in the frequency domain for even better performance estimation in reverberant speech recognition.

Summary updated

BibTeX
@inproceedings{fukumori2010performance,
  author = {Takahiro Fukumori and Masanori Morise and Takanobu Nishiura},
  title = {{Performance estimation of reverberant speech recognition based on reverberant criteria RSR-dn with acoustic parameters}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure