Does Language Matter for Early Detection of Parkinson's Disease from Speech?

Peter Plantinga, Briac Cordelle, Dominique Louër, Mirco Ravanelli, Denise KleinMlsp 2025 · DOI

TL;DR

This paper investigates whether linguistic features are as important as vocal features for detecting Parkinson's Disease (PD) from speech. They found that language-based models, particularly multilingual ones like Whisper, perform comparably to or better than models focusing solely on vocal characteristics, suggesting that language plays a critical role in early PD detection.

What Problem Does This Paper Address?

Diagnosing Parkinson's Disease (PD) can be challenging and time-consuming. Speech analysis offers a promising, low-cost biomarker for early detection and monitoring. However, there's a debate in the research community about whether acoustic (vocal) features or linguistic (language) features are more crucial for accurate PD detection from speech. Early research often focused on sustained vowel phonation (SVP) tasks, which primarily capture vocal characteristics. More recent work explores cognitively demanding tasks that involve language. This paper aims to disentangle the contributions of vocal and linguistic information by systematically evaluating different speech tasks, language models, and encoder types.

Methods

The authors designed experiments to isolate the impact of language versus vocal features. They employed three main strategies:

  1. Varying Speech Tasks: They compared performance on two distinct tasks:
  2. Sustained Vowel Phonation (SVP): Primarily captures vocal quality and articulation.
  3. Picture Description Task (DPT): Requires language production, including syntax, semantics, and word choice.
  4. Varying Language Models: They tested models pretrained on:
  5. Monolingual (English) data: To assess performance when language is limited.
  6. Multilingual data: To see if broader language exposure improves detection. They specifically highlighted the multilingual Whisper model, known for its transcription, translation, and language identification capabilities.
  7. Varying Encoders: They used a range of encoders, including:
  8. Interpretable features: Mel filterbanks, OpenSMILE, and SpeechBrain VocalFeatures as baselines.
  9. Automatic Speech Recognition (ASR) models: Such as Whisper and Crisper Whisper, which process speech into text.
  10. Self-Supervised Learning (SSL) models: Like WavLM, HuBERT, and wav2vec 2.0, trained on either speech or broader audio events (AudioSet).
  11. Text Encoders: Specifically, multilingual BERT and Llama models for transcript analysis.

Crucially, for all experiments, they froze the pretrained models and added minimal parameters for classification. This approach aims to evaluate the inherent capabilities of the pretraining without the confounding effects of extensive fine-tuning.

Datasets

The primary dataset used was from the Quebec Parkinson Network (QPN) [33], comprising speech recordings from 208 PD patients and 52 controls. This dataset was previously used to investigate articulation impairments in PD. The recordings were made using a headset microphone in a quiet room, with most patients in the ON medication state. For evaluation, a demographically balanced test set of 32 subjects was created, ensuring equal representation of patients/controls, native French/English speakers, and men/women. Subjects in this test set were in the early stages of PD (average 3.1 years post-diagnosis) with mild motor symptoms. For the DPT, 101 recordings were manually transcribed, and Google Cloud's Chirp2 model was used for automatic transcription of the rest.

Key Insights and Ideas

  • Language is as informative as voice: The study strongly suggests that linguistic features derived from tasks like picture description are as effective, if not more so, than purely vocal features from SVP tasks for PD detection.
  • Multilingual models offer a significant boost: The multilingual Whisper model significantly outperformed its English-only counterpart, indicating that broader language exposure and multilingual objectives are beneficial for PD detection. This could be due to improved robustness to speech variations or better generalization.
  • Task matters: The Picture Description Task (DPT) consistently yielded better results than the Sustained Vowel Phonation (SVP) task across most systems, highlighting the importance of cognitively demanding, language-rich tasks.
  • AudioSet pretraining helps for vocal tasks: For the SVP task, SSL models pretrained on AudioSet (a broad audio event dataset) performed better than those trained solely on speech. This suggests that when linguistic information is scarce (as in SVP), a more general acoustic understanding is advantageous.
  • SSL models show less language dependence: Compared to Whisper, SSL models exhibited smaller performance differences between monolingual and multilingual training, suggesting they might rely more on general acoustic cues rather than specific linguistic structures.

Key Results

  • DPT outperforms SVP: For most systems, performance on the DPT was significantly higher than on the SVP task, which was often near chance (50%).
  • Text-based models are competitive: The best text-based model achieved a 70% F1 score on the DPT, comparable to the best SVP performance (68%) and the best DPT performance using vocal features (66%).
  • Multilingual Whisper leads: The multilingual Whisper model achieved the highest accuracy, outperforming other tested models. The English-only Whisper performed considerably worse.
  • AudioSet SSL models excel on SVP: SSL models pretrained on AudioSet showed strong performance on the SVP task (e.g., 67.5% F1 for HuBERT Base - AS), outperforming speech-only SSL models.

Limitations and Future Work

  • ASR transcript errors: The study notes that ASR errors, particularly for French transcripts, led to reduced performance for text-based models. This highlights the need for robust ASR or manual transcription for optimal results.
  • Model freezing: While freezing models helps isolate pretraining effects, it might not represent the absolute best performance achievable with fine-tuning.
  • Future Work: The authors suggest that future research should prioritize tasks that engage language areas of the brain and consider the linguistic aspects of model pretraining when selecting foundation models for PD speech biomarkers. They also imply that a combination of linguistic and vocal features would likely yield the best detection performance.

Summary updated

BibTeX
@inproceedings{plantinga2025does,
  author = {Peter Plantinga and Briac Cordelle and Dominique Louër and Mirco Ravanelli and Denise Klein},
  title = {{Does Language Matter for Early Detection of Parkinson's Disease from Speech?}},
  booktitle = {Mlsp},
  year = {2025},
  doi = {10.1109/mlsp62443.2025.11204272},
}
Figure