Automated Speech Markers of Alzheimer Dementia: Test of Cross-Linguistic Generalizability

Paula Andrea Pérez-Toro, F. Ferrante, Gonzalo Pérez, Boon Lead Tee, Jessica Deleon, Elmar Nöth, Maria Schuster, Andreas Maier, Andrea Slachevsky, Maria Luisa Gorno‐Tempini, Agustín Ibáñez, Juan Rafael Orozco‐Arroyave, Adolfo M. GarcíaJournal of Medical Internet Research 2025 · DOI

TL;DR

This study investigates whether interpretable speech and language features can reliably detect Alzheimer's Disease (AD) across English and Spanish speakers. The key finding is that speech timing features, like pause duration and speech rate, generalize well across languages, while lexico-semantic features do not. This suggests that timing-based features are more promising for developing universally applicable automated speech analysis tools for AD detection.

What Problem Does This Paper Address?

Automated Speech and Language Analysis (ASLA) is a promising, non-invasive method for early AD detection. However, current ASLA research faces two major hurdles: 1) many studies use computationally derived features that lack clinical interpretability, making it hard to understand why a model makes a certain prediction, and 2) a significant portion of research is limited to English speakers, hindering the generalizability of findings to the diverse global population affected by AD. This lack of cross-linguistic validity is particularly problematic as AD prevalence is high in non-Anglophone regions. The paper aims to address these gaps by testing interpretable linguistic and acoustic features for AD detection in both within-language (English-English) and between-language (English-Spanish) settings.

Methods

The researchers employed a machine learning approach using decision trees and sparse multilayer perceptrons. They extracted two main types of interpretable features from participants' speech:

  • Speech Timing Features: These include measures related to the rhythm and flow of speech, such as pause duration, speech segment duration ratio, and voice rate. These features are thought to reflect cognitive processes like lexical search effort.
  • Lexico-Semantic Features: These capture aspects of word choice and meaning, including word-class ratios (e.g., proportion of nouns, verbs), semantic granularity (how specific a word is), and semantic variability (how related successive words are in meaning).

The study used a "0-shot" approach for cross-linguistic testing, meaning the model trained on English data was directly applied to Spanish data without any intermediate fine-tuning or calibration. This provides a stringent test of generalizability. Additionally, they explored the correlation between predicted AD status and actual cognitive function as measured by the Mini-Mental State Examination (MMSE).

Datasets

The study utilized two datasets:

  • English Dataset: 117 participants (58 with AD, 59 healthy controls) from the Pitt Corpus, a well-established resource for ASLA research on AD.
  • Spanish Dataset: 94 participants (47 with AD, 47 healthy controls) from a Chilean cohort.

The datasets were matched for sociodemographic variables (age, sex, education), and AD patients were matched for MMSE scores within their respective language groups. The Spanish dataset was used exclusively as an independent testing set for the between-language analysis.

Key Insights and Ideas

  • Interpretable Features Can Be Powerful: The study demonstrates that clinically interpretable features (speech timing and lexico-semantic) can achieve high performance in AD detection, comparable to or even exceeding some "black-box" models.
  • Speech Timing is Language-Agnostic: The most striking finding is that speech timing features show consistent performance in identifying AD across both English and Spanish speakers. This suggests that disruptions in speech rhythm and flow are fundamental markers of AD, less influenced by linguistic variations.
  • Lexico-Semantic Features are Language-Specific: In contrast, lexico-semantic features, while informative within English, did not generalize well to Spanish speakers. This highlights how language-specific nuances in vocabulary and semantics can confound cross-linguistic comparisons.
  • Fusion Doesn't Always Improve Cross-Linguistic Performance: While combining speech timing and lexico-semantic features improved within-language classification, it did not enhance performance in the between-language setting. This further supports the idea that the language-specific nature of lexico-semantic features can dilute the benefits of multimodal approaches when crossing language barriers.
  • Timing Features Predict Cognitive Severity: Speech timing features not only helped classify AD patients but also showed a significant correlation with MMSE scores in both within- and between-language settings, indicating their potential for tracking cognitive decline.

Key Results

  • Within-Language (English):
    • Combined speech timing and lexico-semantic features achieved the highest classification accuracy (AUC = 0.88).
  • Speech timing features alone: AUC = 0.79.
  • Lexico-semantic features alone: AUC = 0.80.
  • Speech timing features significantly predicted MMSE scores (p < .001).
  • Between-Language (English to Spanish):
    • Speech timing features showed good classification performance (AUC = 0.75).
  • Lexico-semantic features performed poorly (AUC = 0.64).
  • Fusion of both feature types also yielded limited performance (AUC = 0.65).
  • Speech timing features significantly predicted Spanish-speaking patients' MMSE scores (p < .001).

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Acquisition Differences: The English and Spanish datasets had slightly different audio acquisition conditions, although efforts were made to harmonize them.
  • Diagnostic Criteria: The English dataset's AD diagnoses predated current criteria, though patient selection was based on consistent MMSE scores and clinical decline.
  • Sample Size: The Spanish cohort was relatively small, limiting statistical power for cross-linguistic analyses.
  • Intervention Data: Information on examiner interventions during speech tasks was not fully captured.
  • Cross-Sectional Design: The study captures current cognitive status, not the progression of AD over time.
  • Denoising Impact: While denoising improves performance, its potential to introduce artifacts warrants further investigation.

Future work should focus on:

  • Replicating findings with larger, more harmonized cross-linguistic datasets.
  • Investigating bidirectional generalizability (Spanish-to-English).
  • Exploring hybrid approaches that combine the robustness of interpretable features with the potential gains of few-shot learning or transfer learning.
  • Comparing interpretable features with embedding-based approaches to understand the trade-offs between interpretability and sensitivity.
  • Investigating whether speech timing anomalies reflect purely cognitive deficits or also motor speech dysfunction.

Summary updated

Abstract

These results suggest that while both timing and lexico-semantic features are informative within the same language, only speech timing features demonstrate consistent performance across languages. By focusing on clinically interpretable features, this approach supports the development of clinically usable ASLA tools.
BibTeX
@article{pereztoro2025automated,
  author = {Paula Andrea Pérez-Toro and F. Ferrante and Gonzalo Pérez and Boon Lead Tee and Jessica Deleon and Elmar Nöth and Maria Schuster and Andreas Maier and Andrea Slachevsky and Maria Luisa Gorno‐Tempini and Agustín Ibáñez and Juan Rafael Orozco‐Arroyave and Adolfo M. García},
  title = {{Automated Speech Markers of Alzheimer Dementia: Test of Cross-Linguistic Generalizability}},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  doi = {10.2196/74200},
}
Figure