Abstract
Automated screening for Alzheimer’s Disease (AD) typically relies on the linguistic analysis of patient speech. While manual transcriptions provide high accuracy, scalable screening necessitates the use of Automatic Speech Recognition (ASR). This paper investigates the "ASR-robustness" paradox: the observation that AD detection models maintain high performance even when ASR systems produce significant Word Error Rates (WER). Through an analysis of error distributions and embedding stability, the study demonstrates that ASR systems primarily fail on diagnostically insignificant stopwords while preserving the task-related keywords essential for clinical assessment.
Introduction
In the field of medical AI, the transition from manual to automated transcription is often hindered by the "disordered speech" characteristic of cognitive impairment. Patients with Alzheimer’s Disease frequently exhibit hesitations, repetitions, and phonetic deviations that challenge standard ASR systems. However, researchers have noted a non-linear relationship between ASR quality and downstream diagnostic accuracy. A system with a 30% WER can often achieve diagnostic parity with a 0% WER manual transcript. This study seeks to explain this phenomenon by categorizing ASR errors and measuring their impact on transformer-based classification models.
Methodology
The researchers developed a robust pipeline to evaluate the impact of transcription errors on AD detection:
- ASR System: A Time Delay Neural Network (TDNN) was trained on 1,000 hours of Librispeech data and subsequently fine-tuned on the Pitt Corpus to ensure in-domain sensitivity.
- Feature Extraction: The model utilized BERT (Bidirectional Encoder Representations from Transformers) to extract semantic embeddings from the [CLS] token of the transcripts.
- Dimensionality Reduction and Classification: To mitigate overfitting on a relatively small clinical dataset, Principal Component Analysis (PCA) reduced the 768-dimensional BERT embeddings to 108 dimensions. A Support Vector Machine (SVM) with a linear kernel served as the final classifier.
- Ablation Analysis: The team performed "word ablation" experiments, systematically removing or substituting stopwords versus task-related keywords to observe the shift in BERT embeddings relative to the SVM’s decision boundary (the "hyperplane offset").
Dataset
The study utilized the ADReSS (Alzheimer’s Dementia Recognition Through Spontaneous Speech) Challenge 2020 corpus, a standardized subset of the Pitt Corpus from the DementiaBank database. The task involved 156 English speakers (78 healthy controls and 78 AD patients) describing the "Cookie Theft" picture. The data was split into 108 speakers for training and 48 for testing.
Analysis of Error Distribution
The investigation revealed that ASR errors are not distributed uniformly across the lexicon. Instead, they follow a "long-tail" distribution where a small subset of words accounts for the majority of errors.
- Stopword Dominance: Approximately 60% of all ASR errors occurred on stopwords (e.g., the, is, and, to). These words are often short and phonetically reduced in natural speech, making them difficult for ASR to capture but also making them less relevant for semantic analysis.
- Keyword Resilience: In contrast, task-related keywords (e.g., cookie, jar, sink, overflow) were recognized with much higher reliability. While the overall WER was 33.9%, the error rate for these critical keywords was only 14.3%. Keywords tend to be longer and phonetically distinct, which aids the ASR system in correct identification.
Embedding Stability and Diagnostic Weight
The study found that healthy participants mentioned task-related keywords twice as often as AD patients, confirming their role as primary diagnostic markers. BERT embeddings proved remarkably robust to the noise introduced by stopword errors. In ablation tests, removing 100% of stopwords shifted the embedding toward the decision boundary but rarely resulted in a misclassification. Conversely, the model was highly sensitive to keywords; removing more than 70% of keywords caused the embeddings to cross the decision boundary, leading to diagnostic failure.
Results and Discussion
Despite a high overall WER of 33.9%, the ASR-based detection system achieved an accuracy of 88%, which was identical to the performance achieved using perfect manual transcriptions. This suggests that the BERT-SVM pipeline effectively ignores the "noise" of stopword errors and focuses on the semantic presence of keywords and broader linguistic structures.
However, the study notes limitations. The current analysis focuses primarily on word-level errors within a constrained task (the "Cookie Theft" description). Future research is required to determine if this robustness holds in open-ended conversations where syntactic dependencies and discourse coherence—factors also affected by AD—play a larger role in diagnosis.
Conclusion
The findings suggest that for AD detection, the quality of ASR should not be judged by a global Word Error Rate alone. Because ASR systems are more likely to preserve the phonetically rich keywords that carry the highest diagnostic weight, they remain a viable and effective tool for scalable, automated Alzheimer’s screening.
Summary updated