On the Use of Pitch Features for Disordered Speech Recognition

Shansong Liu, Shoukang Hu, Xunying Liu, Helen Meng

Introduction

Automatic Speech Recognition (ASR) for individuals with speech disorders, such as dysarthria, presents significant challenges due to degraded voice quality and uncoordinated muscle movements. While pitch is a valuable prosodic cue for typical speakers—especially in tonal languages—it is often distorted in disordered speech, manifesting as "pitch breaks" or abnormal frequencies. In standard ASR systems, pitch features are typically concatenated with acoustic features. However, for dysarthric speakers, these features can act as noise rather than helpful signals. This paper investigates robust methods for integrating pitch features using Gated Neural Networks (GNN) and Bayesian GNNs (BGNN) to intelligently weight pitch information based on its reliability.

Methodology

The researchers proposed a selection mechanism to dynamically integrate pitch features. Three primary architectures were evaluated:

  1. Standard DNN: Acoustic filter banks and pitch features are concatenated at the input layer.
  2. Gated Neural Network (GNN): A dedicated gating layer acts as a learnable filter, weighting pitch features before they enter the network. This allows the system to "turn down" unreliable pitch data.
  3. Bayesian Gated Neural Network (BGNN): To address overfitting on small, variable datasets, the BGNN treats gating parameters as distributions rather than fixed values. This Bayesian approach models the uncertainty inherent in noisy pitch data.
Architecture of the proposed Gated Neural Network (GNN) and Bayesian GNN (BGNN) showing the gating layer for pitch feature selection.

Pitch extraction was performed using the Kaldi pitch tracker, providing a 3-dimensional vector: Probability of Voicing (POV), normalized pitch, and delta-log-pitch.

Experimental Setup

The methods were tested on two corpora:

  • UASpeech (English): An isolated word recognition task involving 16 dysarthric speakers with varying severity levels.
  • CUDYS (Cantonese): A Cantonese dysarthric speech corpus. As a tonal language, Cantonese relies heavily on pitch for linguistic meaning, making robust pitch integration particularly critical.

Results and Discussion

The experimental results demonstrate that dynamic weighting is essential for handling the inconsistent quality of pitch in disordered speech.

UASpeech Performance

The BGNN achieved the most significant improvements, reaching an average Word Error Rate (WER) of 31.0%. This represents an 8.3% relative reduction in error compared to a baseline DNN that excluded pitch features. The Bayesian approach proved superior by modeling the reliability of the gating weights, which is crucial for the small and highly variable datasets typical of medical speech studies. Visual analysis of gating heatmaps further revealed that the BGNN was more decisive than the standard GNN, focusing heavily on the first two dimensions of pitch information (POV and normalized pitch) to improve recognition.

Word Error Rate (WER%) results for various models on the UASpeech dataset across different dysarthria types.

CUDYS Performance

For Cantonese speakers with Cerebral Palsy (CP), the GNN provided a 1.3% absolute WER reduction over standard concatenation. This highlights that in tonal languages, where the "cost" of ignoring pitch is high, an intelligent gating mechanism is vital to recover linguistic information without introducing noise.

Conclusion

This research demonstrates that while pitch features are a "double-edged sword" for disordered speech recognition, they can be effectively harnessed through intelligent gating. The BGNN, in particular, provides a robust framework for managing uncertainty in noisy data. Future work will focus on addressing speaker-specific variability and optimizing the computational complexity of Bayesian models for real-time applications.

Summary updated

Abstract

Pitch features have long been known to be useful for recognition of normal speech. However, for disordered speech, the significant degradation of voice quality renders the prosodic features, such as pitch, not always useful, particularly when the underlying conditions, for example, damages to the cerebellum, introduce a large effect on prosody control. Hence, both acoustic and prosodic information can be distorted. To the best of our knowledge, there has been very limited research on using pitch features for disordered speech recognition. In this paper, a comparative study of multiple approaches designed to incorporate pitch features is conducted to improve the performance of two disordered speech recognition tasks: English UASpeech, and Cantonese CUDYS. A novel gated neural network (GNN) based approach is used to improve acoustic and pitch feature integration over a conventional concatenation between the two. Bayesian estimation of GNNs is also investigated to further improve their robustness.
BibTeX
@inproceedings{liu2019use,
  author = {Shansong Liu and Shoukang Hu and Xunying Liu and Helen Meng},
  title = {{On the Use of Pitch Features for Disordered Speech Recognition}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-2609},
}
Figure