Cross-Lingual Alzheimer’s Disease Detection Based on Paralinguistic and Pre-Trained Features

Xuchu Chen, Yu Pu, Jinpeng Li, Wei-Qiang Zhang

TL;DR

This paper tackles the challenge of detecting Alzheimer's Disease (AD) from speech across different languages, a crucial step for early diagnosis. They found that traditional paralinguistic speech features, which capture vocal nuances like pauses and tone, performed better than modern pre-trained models for this cross-lingual task, suggesting these features are more robust to language differences.

What Problem Does This Paper Address?

Alzheimer's Disease (AD) is a neurodegenerative condition that affects cognitive and expressive abilities. Early detection is vital for managing the disease. While detecting AD from speech has shown promise in single languages, the ability to do so across different languages (cross-lingual detection) is less understood. This is particularly challenging because speech patterns and language structures vary significantly. The ICASSP-SPGC-2023 ADReSS-M Challenge specifically aims to bridge this gap by requiring models trained on English speech data to perform AD detection on Greek speech data. This paper investigates which types of speech features can generalize effectively across languages for AD detection, addressing the lack of a unified understanding in this area.

Methods

The authors propose a method that leverages two main types of speech features for cross-lingual AD detection:

  • Paralinguistic Features: These features capture the non-linguistic aspects of speech, such as pitch, intensity, speaking rate, and voice quality. They are extracted using the openSmile toolkit, specifically using the IS10-Paralinguistics-compat, IS10-Paralinguistics, and IS11-speaker-state feature sets. These features are then fed into a Support Vector Machine (SVM) for classification and regression.
  • Pre-trained Features: This approach utilizes powerful, pre-trained models to extract rich representations from speech.
  • Acoustic Features: The XLSR-53 model, a cross-lingual model trained on speech data from 53 languages, is used to extract acoustic features. These features are then processed by fully connected (FC) layers.
  • Linguistic Features: Speech is first transcribed into text using the Whisper model. Then, the RoBERTa model, a pre-trained language model, is fine-tuned on these transcripts to extract linguistic features. These features are also processed by FC layers.

The system is evaluated on two tasks: classifying individuals as AD patients or healthy controls, and predicting the Mini Mental State Examination (MMSE) score.

Datasets

The paper utilizes data from the ICASSP-SPGC-2023 ADReSS-M Challenge. The training set for this challenge is in English, while the test set is in Greek. The specific datasets are not detailed beyond this language mismatch.

Key Insights and Ideas

  • Paralinguistic Features Shine in Cross-Lingual Settings: The study's most significant finding is that traditional paralinguistic features, extracted by openSmile, outperformed modern pre-trained models (XLSR-53 for acoustics and RoBERTa for linguistics) in the cross-lingual AD detection task.
  • Robustness to Language: The authors hypothesize that paralinguistic features are more effective because they can capture subtle speech changes (like unexpected pauses) that are less dependent on the specific language being spoken. This makes them more generalizable across linguistic boundaries.
  • Pre-trained Models' Language Dependency: Conversely, the high-dimensional embeddings from pre-trained models might be more tightly coupled with the specific language they were trained on, leading to weaker generalization in a cross-lingual scenario, especially if the target language (Greek in this case) was not well-represented during their pre-training.
  • Potential for Early AD Detection: The successful regression task (predicting MMSE scores) with an RMSE of 4.788 suggests that speech analysis can indeed provide valuable insights for monitoring AD progression, even across languages.

Key Results

  • Classification Task: The best performance in the classification task (distinguishing AD patients from healthy individuals) was achieved by a paralinguistic feature-based method (Method 2, IS10-Paralinguistics + SVM), yielding an F1 score of 0.72. Other paralinguistic methods also showed competitive results.
  • Pre-trained Feature Performance: The method using XLSR-53 acoustic features and FC layers achieved an F1 score of 0.6441. The method using Whisper for transcription and RoBERTa for linguistic features had a high precision but a lower recall, resulting in a lower F1 score.
  • Regression Task: The proposed method achieved a Root Mean Squared Error (RMSE) of 4.788 for the MMSE score prediction task, indicating its effectiveness in quantifying AD severity from speech.

Limitations and Future Work

  • Limitations: The authors acknowledge that the accuracy of the XLSR-53 based method was not as high as expected, potentially due to the absence of Greek data in its training set. They also note that the performance of the Whisper model could have impacted the linguistic feature results. A general limitation is the observed weaker generalization of pre-trained models in this specific cross-lingual AD detection context.
  • Future Work: The authors plan to integrate paralinguistic and pre-trained features to potentially achieve even better performance. They also intend to explore ensemble models that combine different feature types to enhance cross-lingual AD detection capabilities.

Summary updated

Abstract

We present our submission to the ICASSP-SPGC-2023 ADReSS-M Challenge Task, which aims to investigate which acoustic features can be generalized and transferred across languages for Alzheimer’s Disease (AD) prediction. The challenge consists of two tasks: one is to classify the speech of AD patients and healthy individuals, and the other is to infer Mini Mental State Examination (MMSE) score based on speech only. The difficulty is mainly embodied in the mismatch of the dataset, in which the training set is in English while the test set is in Greek. We extract paralinguistic features using openSmile toolkit and acoustic features using XLSR-53. In addition, we extract linguistic features after transcribing the speech into text. These features are used as indicators for AD detection in our method. Our method achieves an accuracy of 69.6% on the classification task and a root mean squared error (RMSE) of 4.788 on the regression task. The results show that our proposed method is expected to achieve automatic multilingual Alzheimer’s Disease detection through spontaneous speech.
BibTeX
@misc{chen2023cross,
  author = {Xuchu Chen and Yu Pu and Jinpeng Li and Wei-Qiang Zhang},
  title = {{Cross-Lingual Alzheimer’s Disease Detection Based on Paralinguistic and Pre-Trained Features}},
  year = {2023},
  eprint = {2303.07650},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.07650},
}
Figure