Automatic Speech Recognition Performance Improvement for Mandarin Based on Optimizing Gain Control Strategy

Desheng Wang, Yangjie Wei, Ke Zhang, Dong Ji, Yi WangSensors 2022 · DOI

TL;DR

This paper introduces a novel "Maximized Original Signal Transmission" (MOST) gain control strategy for Automatic Speech Recognition (ASR) systems. By theoretically modeling gain control's impact on ASR performance using noise figure analysis, they demonstrate that strategically placing gain control operations at the end of the signal chain significantly reduces Word Error Rate (WER), achieving up to a 10% improvement on Mandarin speech.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems are increasingly integrated into our daily lives, powering voice assistants, smart devices, and more. However, their performance, especially in real-world, noisy environments, can be suboptimal. One crucial but often overlooked factor affecting ASR accuracy is "gain control," which adjusts the amplitude of audio signals. While necessary for managing signal levels and preventing clipping, current ASR systems often employ unconstrained gain control strategies. This lack of theoretical grounding means these strategies can inadvertently introduce noise and distort speech features, leading to a higher Word Error Rate (WER) – essentially, the system misunderstands more words. The paper aims to bridge this gap by providing a theoretical framework and a practical solution to optimize gain control for improved ASR performance.

Methods

The core of this research lies in a two-pronged approach: theoretical modeling and experimental validation.

  1. Theoretical Modeling using Noise Figure: The authors draw an analogy between ASR signal processing chains and Radio Frequency (RF) signal chains. They leverage the concept of "noise figure," a standard metric in RF engineering that quantifies how much a component degrades the signal-to-noise ratio (SNR). By modeling ASR units (receiving, middle, and recognition) as cascaded components in an RF chain, they analyze how different gain control placements affect the overall noise figure. The key insight here is that placing gain control operations later in the signal processing pipeline, closer to the recognition unit, minimizes the cumulative noise introduced.
  2. Proposed MOST Gain Control Strategy: Based on the noise figure analysis, they propose the "Maximized Original Signal Transmission" (MOST) strategy. This strategy enforces a constraint: gain control operations should be performed as close to the recognition unit as possible. This means if multiple gain adjustments are needed, they should ideally be consolidated at the very end of the signal processing chain, rather than distributed across various intermediate stages.
  3. Experimental Validation: To prove the effectiveness of MOST, they conducted experiments using a practical ASR scenario (a smart voice recorder) and a Mandarin dataset. They compared the proposed MOST strategy against current, unconstrained gain control methods under various noise and reverberation conditions.

Datasets

The experiments were conducted using a subset of the THCHS-30 dataset, a Mandarin speech corpus. This dataset consists of 500 test utterances from 10 native Mandarin speakers, aged 19-50, with a sampling frequency of 16 kHz and 16-bit resolution. The paper does not introduce a new benchmark dataset but rather uses a standard one for evaluation.

Key Insights and Ideas

  • Gain Control is Not Just About Volume: The paper highlights that gain control isn't merely about setting a comfortable listening volume; it's a critical signal processing step that can significantly impact the fidelity of speech features.
  • Location, Location, Location: The placement of gain control operations within the ASR pipeline is far more important than previously appreciated. Pushing gain control towards the end of the chain is the key to minimizing its negative impact.
  • Noise Figure as a Unifying Metric: The application of RF noise figure theory to ASR signal processing provides a powerful, quantitative way to analyze and optimize gain control strategies.
  • Theoretical Proof of Optimality: The paper offers a theoretical proof that the MOST strategy, by minimizing the overall noise figure, leads to optimal ASR performance under gain control constraints.

Key Results

The experimental results demonstrate the effectiveness of the MOST gain control strategy:

  • Significant WER Reduction: The proposed MOST strategy consistently outperformed current gain control strategies across various noise and reverberation conditions.
  • Average 10% WER Reduction: Under a -9 dB gain condition, the MOST strategy achieved an average absolute WER reduction of approximately 10% on the Mandarin dataset.
  • Impact of Gain Level: The performance improvement was more pronounced with higher gain attenuation (e.g., -9 dB) compared to lower attenuation (e.g., -3 dB), where the improvement was around 2%.
  • Robustness in Harsh Conditions: While performance degraded for both strategies in extremely noisy or reverberant environments, the MOST strategy showed a more gradual decline, indicating better robustness.

Limitations and Future Work

  • Dataset Limitation: The experiments were conducted on the Meridian dataset, which is a limitation. The authors suggest that future work should involve proving the effectiveness of the MOST strategy with other languages.
  • Signal Layer Focus: The paper focuses on the "signal layer" of ASR, optimizing gain control at the audio input stage. Future work could explore other signal layer aspects that affect ASR performance, such as dynamic range compression at the microphone.
  • Generalizability: While the paper suggests the MOST strategy is likely applicable to other languages due to its focus on speech signal features, this remains an area for future investigation.

Summary updated

Abstract

Automatic speech recognition (ASR) is an essential technique of human-computer interactions; gain control is a commonly used operation in ASR. However, inappropriate gain control strategies can lead to an increase in the word error rate (WER) of ASR. As there is a current lack of sufficient theoretical analyses and proof of the relationship between gain control and WER, various unconstrained gain control strategies have been adopted on realistic ASR systems, and the optimal gain control with respect to the lowest WER, is rarely achieved. A gain control strategy named maximized original signal transmission (MOST) is proposed in this study to minimize the adverse impact of gain control on ASR systems. First, by modeling the gain control strategy, the quantitative relationship between the gain control strategy and the ASR performance was established using the noise figure index. Second, through an analysis of the quantitative relationship, an optimal MOST gain control strategy with minimal performance degradation was theoretically deduced. Finally, comprehensive comparative experiments on a Mandarin dataset show that the proposed MOST gain control strategy can significantly reduce the WER of the experimental ASR system, with a 10% mean absolute WER reduction at -9 dB gain.
BibTeX
@inproceedings{wang2022automatic,
  author = {Desheng Wang and Yangjie Wei and Ke Zhang and Dong Ji and Yi Wang},
  title = {{Automatic Speech Recognition Performance Improvement for Mandarin Based on Optimizing Gain Control Strategy}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22083027},
}
Figure