Study of the Performance of Automatic Speech Recognition Systems in Speakers with Parkinson's Disease

Laureano Moro-Velázquez, Jaejin Cho, Shinji Watanabe, Mark A. Hasegawa-Johnson, Odette Scharenborg, Heejin Kim, Najim Dehak

Introduction

For individuals living with Parkinson’s Disease (PD), motor impairments often make traditional typing difficult, positioning voice-controlled assistive technology as a critical tool for maintaining independence. However, PD is frequently accompanied by hypokinetic dysarthria—a speech disorder characterized by reduced volume, imprecise articulation, and irregular speech "rushes." This study quantifies the performance gap of modern Automatic Speech Recognition (ASR) systems when processing PD-affected speech, identifying a significant increase in Word Error Rate (WER) compared to healthy controls.

Methodology

The research evaluates two state-of-the-art ASR architectures for the Spanish language:

  1. End-to-End (E2E): An attention-based model utilizing Connectionist Temporal Classification (CTC) loss to map acoustic frames directly to character sequences.
  2. Hybrid HMM/DNN: A traditional pipeline employing a Resnet-style Time Delay Neural Network (TDNN) to predict phonetic states within a Hidden Markov Model framework.

Beyond standard accuracy metrics, the authors conducted a detailed phonetic error analysis, examining "Manner Classes" (e.g., plosives, nasals) and correlating ASR performance with clinical metrics like the Unified Parkinson’s Disease Rating Scale (UPDRS).

Datasets

The models were trained on the Fisher Spanish (FisherSP) corpus, consisting of 163 hours of telephone conversations. Evaluation was performed using the Neurovoz dataset, which includes 43 speakers with PD and 46 healthy controls (HC). Participants provided 15 fixed sentences, and the dataset includes clinical metadata such as years since diagnosis (YSD) and disease severity scores.

Phonetic Analysis and Insights

The study revealed specific phonetic degradations unique to Parkinsonian speech. A primary finding was the "spirantization" effect, where motor control limitations prevent speakers from fully closing the vocal tract for "stop" consonants (e.g., /p/, /b/), causing them to be perceived as fricatives. Additionally, the vowel /u/ showed extreme vulnerability, with substitution rates 12 times higher in PD patients than in healthy controls.

Phonetic confusion matrices for HC and PD speakers across two ASR architectures, highlighting the increased phonetic confusion in the PD group.

The E2E system also frequently "de-accented" Spanish vowels, likely due to the impaired prosody and pitch control associated with the disease. Clinical correlation analysis showed that WER was moderately correlated with time since diagnosis (ρ = 0.47), suggesting that acoustic distance grows predictably as the disease progresses.

Key Results

The analysis demonstrated a clear performance gap. In the E2E system, healthy controls achieved a WER of 36.7%, while PD patients' WER rose to 47.0%. The Hybrid system showed a similar trend (39.4% vs 47.3%).

WER distributions for ASR 1 and ASR 2, illustrating the significant shift toward higher error rates for PD speakers compared to healthy controls (HC).

PD speakers exhibited significantly higher rates of deletions and insertions, attributed to "speech rushes" and disprosody. While the E2E model performed better on healthy speech, both architectures degraded to similar levels when processing dysarthric speech, suggesting that current modern architectures lack inherent robustness to these specific acoustic variations.

Conclusion

While general trends were identified, the highly individual nature of PD-affected speech suggests that "one-size-fits-all" ASR models may be insufficient. Future research should focus on personalized ASR adaptation techniques, detailed analysis of points of articulation, and the evaluation of spontaneous, conversational speech to better support the needs of this population.

Summary updated

Abstract

Parkinson’s Disease (PD) affects motor capabilities of patients, who in some cases need to use human-computer assistive technologies to regain independence. The objective of this work is to study in detail the differences in error patterns from state-of-the-art Automatic Speech Recognition (ASR) systems on speech from people with and without PD. Two different speech recognizers (attention-based end-to-end and Deep Neural Network - Hidden Markov Models hybrid systems) were trained on a Spanish language corpus and subsequently tested on speech from 43 speakers with PD and 46 without PD. The differences related to error rates, substitutions, insertions and deletions of characters and phonetic units between the two groups were analyzed, showing that the word error rate is 27% higher in speakers with PD than in control speakers, with a moderated correlation between that rate and the developmental stage of the disease. The errors were related to all manner classes, and were more pronounced in the vowel /u/. This study is the first to evaluate ASR systems’ responses to speech from patients at different stages of PD in Spanish. The analyses showed general trends but individual speech deficits must be studied in the future when designing new ASR systems for this population.
BibTeX
@inproceedings{morovelazquez2019study,
  author = {Laureano Moro-Velázquez and Jaejin Cho and Shinji Watanabe and Mark A. Hasegawa-Johnson and Odette Scharenborg and Heejin Kim and Najim Dehak},
  title = {{Study of the Performance of Automatic Speech Recognition Systems in Speakers with Parkinson's Disease}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-2993},
}
Figure