Introduction
Automatic Speech Recognition (ASR) for individuals with speech disorders, such as dysarthria, presents significant challenges due to degraded voice quality and uncoordinated muscle movements. While pitch is a valuable prosodic cue for typical speakers—especially in tonal languages—it is often distorted in disordered speech, manifesting as "pitch breaks" or abnormal frequencies. In standard ASR systems, pitch features are typically concatenated with acoustic features. However, for dysarthric speakers, these features can act as noise rather than helpful signals. This paper investigates robust methods for integrating pitch features using Gated Neural Networks (GNN) and Bayesian GNNs (BGNN) to intelligently weight pitch information based on its reliability.
Methodology
The researchers proposed a selection mechanism to dynamically integrate pitch features. Three primary architectures were evaluated:
- Standard DNN: Acoustic filter banks and pitch features are concatenated at the input layer.
- Gated Neural Network (GNN): A dedicated gating layer acts as a learnable filter, weighting pitch features before they enter the network. This allows the system to "turn down" unreliable pitch data.
- Bayesian Gated Neural Network (BGNN): To address overfitting on small, variable datasets, the BGNN treats gating parameters as distributions rather than fixed values. This Bayesian approach models the uncertainty inherent in noisy pitch data.
Pitch extraction was performed using the Kaldi pitch tracker, providing a 3-dimensional vector: Probability of Voicing (POV), normalized pitch, and delta-log-pitch.
Experimental Setup
The methods were tested on two corpora:
- UASpeech (English): An isolated word recognition task involving 16 dysarthric speakers with varying severity levels.
- CUDYS (Cantonese): A Cantonese dysarthric speech corpus. As a tonal language, Cantonese relies heavily on pitch for linguistic meaning, making robust pitch integration particularly critical.
Results and Discussion
The experimental results demonstrate that dynamic weighting is essential for handling the inconsistent quality of pitch in disordered speech.
UASpeech Performance
The BGNN achieved the most significant improvements, reaching an average Word Error Rate (WER) of 31.0%. This represents an 8.3% relative reduction in error compared to a baseline DNN that excluded pitch features. The Bayesian approach proved superior by modeling the reliability of the gating weights, which is crucial for the small and highly variable datasets typical of medical speech studies. Visual analysis of gating heatmaps further revealed that the BGNN was more decisive than the standard GNN, focusing heavily on the first two dimensions of pitch information (POV and normalized pitch) to improve recognition.
CUDYS Performance
For Cantonese speakers with Cerebral Palsy (CP), the GNN provided a 1.3% absolute WER reduction over standard concatenation. This highlights that in tonal languages, where the "cost" of ignoring pitch is high, an intelligent gating mechanism is vital to recover linguistic information without introducing noise.
Conclusion
This research demonstrates that while pitch features are a "double-edged sword" for disordered speech recognition, they can be effectively harnessed through intelligent gating. The BGNN, in particular, provides a robust framework for managing uncertainty in noisy data. Future work will focus on addressing speaker-specific variability and optimizing the computational complexity of Bayesian models for real-time applications.
Summary updated