A Multitask Learning Approach to Assess the Dysarthria Severity in Patients with Parkinson's Disease

Juan Camilo Vásquez Correa, Tomas Arias, Juan Rafael Orozco-Arroyave, Elmar NöthInterspeech 2018

TL;DR

This paper introduces a novel multitask learning approach using convolutional neural networks (CNNs) to simultaneously assess multiple speech impairments in Parkinson's Disease (PD) patients. By training a single model to tackle eleven related tasks, the researchers achieved improved generalization and accuracy compared to individual models, particularly for articulation-related deficits.

What Problem Does This Paper Address?

Parkinson's Disease (PD) is a neurodegenerative disorder that significantly impacts motor control, including speech. This leads to a condition called hypokinetic dysarthria, characterized by a range of speech impairments such as reduced loudness, imprecise articulation, and monotone speech. While previous research has focused on assessing individual speech deficits or classifying PD patients from healthy individuals, there's a need for a more holistic approach that can simultaneously evaluate various aspects of dysarthria. This paper aims to bridge this gap by developing a multitask learning framework that can assess multiple speech impairments concurrently, offering a more comprehensive understanding of the disease's impact on communication.

Methods

The core of this research lies in a multitask learning framework built upon convolutional neural networks (CNNs). The CNN architecture features shared convolutional and pooling layers across all tasks, followed by individual fully connected layers and output layers for each specific task. This shared structure is designed to encourage the network to learn more generalized and representative feature maps.

The input to the CNNs consists of time-frequency representations derived from speech signals. Specifically, the researchers focus on transition modeling, analyzing the moments when vocal fold vibration starts (onset) and stops (offset). These transitions are segmented, and a 160ms chunk of audio around each transition is transformed into a time-frequency representation using the short-time Fourier transform.

The multitask learning loss function is a linear combination of individual loss functions for each task, weighted by hyper-parameters. This allows the model to balance the learning across different speech aspects. The model is trained using stochastic gradient descent with RMSprop for learning rate adaptation, ReLU activation functions, dropout for regularization, and batch normalization for faster training.

Datasets

The study utilizes the PC-GITA database, which contains speech recordings from 50 PD patients and 50 healthy control (HC) Colombian native speakers, balanced for age and gender. Participants produced various utterances, including rapid syllable repetitions, isolated sentences, read text, and monologues.

The speech impairments are evaluated using a modified version of the Frenchay Dysarthria Assessment (m-FDA) scale, which assesses aspects like respiration, lip movement, palate/velum movement, laryngeal function, tongue movement, monotonicity, and intelligibility. The paper also considers standard tasks like classifying PD vs. HC subjects, and assessing neurological state using the MDS-UPDRS-III score.

Key Insights and Ideas

  • Multitask learning enhances generalization: By training a single CNN to perform multiple related speech assessment tasks, the model learns more robust and representative feature maps compared to training individual CNNs for each task.
  • Articulation is key: The multitask learning approach showed the most significant improvements for tasks related to articulation (lips, tongue, larynx movement). This suggests that the transition modeling is particularly effective for capturing these motor control deficits.
  • Holistic assessment is beneficial: Evaluating multiple speech deficits simultaneously provides a more comprehensive picture of dysarthria in PD patients.
  • Transition modeling is a powerful feature: Focusing on the onset and offset of vocal fold vibration proves to be a valuable approach for capturing articulatory capabilities.

Key Results

The multitask learning approach demonstrated improvements in accuracy and unweighted average recall (UAR) across several tasks compared to single-task learning. For instance:

  • PD vs. HC classification: UAR improved from 79% to 89% for offset transitions.
  • m-FDA tongue assessment: UAR increased from 40.7% to 53.8%.
  • m-FDA larynx evaluation: UAR rose from 34.9% to 42.6%.

Overall, the average UAR across all tasks was higher for the multitask approach (49.4%) compared to single-task learning (43.9%). The paper also highlights that the multitask learning scheme provides better generalization, with a smaller reduction in performance between development and test sets.

Limitations and Future Work

The authors acknowledge that the transition modeling approach, while effective for articulation, might be suboptimal for tasks less directly related to articulation, such as respiration, monotonicity, and intelligibility. They suggest that further experiments could explore more specific models for these aspects. Future work could also involve investigating the impact of using only tasks related to articulation capabilities within the multitask framework to potentially further enhance results. Additionally, the m-FDA scale is noted to be in the process of clinical validation.

Summary updated

BibTeX
@inproceedings{correa2018multitask,
  author = {Juan Camilo Vásquez Correa and Tomas Arias and Juan Rafael Orozco-Arroyave and Elmar Nöth},
  title = {{A Multitask Learning Approach to Assess the Dysarthria Severity in Patients with Parkinson's Disease}},
  booktitle = {Interspeech},
  year = {2018},
}
Figure