Automatic Prediction of Linguistic Decline in Writings of Subjects with Degenerative Dementia

Davy Weissenbacher, Travis A. Johnson, Laura Wojtulewicz, Amylou Dueck, Dona Locke, Richard Caselli, Graciela GonzalezNAACL 2016

TL;DR

This paper introduces a Natural Language Processing (NLP) approach to automatically detect early signs of cognitive decline, specifically pre-dementia conditions like Alzheimer's, by analyzing written descriptions of an image. The system achieved an impressive 86.1% accuracy in distinguishing between normal aging and cognitive impairment, suggesting that linguistic irregularities in writing can serve as a powerful biomarker.

What Problem Does This Paper Address?

Alzheimer's disease and other dementias are a growing global concern, and early detection is crucial for potential interventions to slow disease progression. While clinical methods exist, they are often applied late in the disease's course and can be costly. Families often notice subtle changes in language use before formal diagnosis. Existing linguistic analysis methods for detecting cognitive decline, particularly in conversational speech, can be subjective and require expert annotation. This paper aims to bridge this gap by developing an automated system using NLP to analyze written language samples, specifically descriptive writing, for early detection of cognitive decline.

Methods

The researchers developed a pipeline that processes written descriptions using various NLP techniques. Key steps include:

  • Tokenization and Lemmatization: Breaking down text into words and reducing them to their base form.
  • Part-of-Speech Tagging: Identifying the grammatical role of each word (noun, verb, adjective, etc.).
  • N-gram Analysis: Examining sequences of words (or characters) to capture stylistic and syntactic patterns. They specifically used character N-grams due to the small corpus size.
  • Lexical Features: Analyzing ratios of different word types (e.g., adjective/noun ratio) and vocabulary richness (Type-Token Ratio) to detect potential impoverishment of language.
  • Stylometric Features: Measuring text length (document, sentence), and using metrics like Brunét's Index and Honoré's Statistic, which are less sensitive to text length and can indicate vocabulary reduction.
  • Semantic Features (Word2Vec Distance): This is a novel contribution. They created a set of "generative words" from descriptions written by normal subjects. The hypothesis is that subjects in decline will use fewer of these generative words and more inappropriate words, leading to a larger "Word2Vec Distance" (a measure of semantic dissimilarity). They used pre-trained Word2Vec models on a large Google News dataset.
  • Classifier: A Bayesian Network (BN) was used for classification, chosen for its ability to learn complex decision functions, interpretability for medical experts, and fast training. They employed a leave-one-out cross-validation strategy due to the small dataset size.

Datasets

The study utilized a unique corpus of 201 handwritten descriptions of a single control image, collected from participants of the Arizona Alzheimer's Disease Center (ADC) longitudinal study. This corpus included:

  • 154 descriptions from healthy subjects.
  • 47 descriptions from subjects diagnosed with MCI or other forms of degenerative dementia (labeled as "Declined").

This corpus is noted as being unique in its nature and size for this type of linguistic analysis.

Key Insights and Ideas

  • Written descriptions are a rich source of linguistic biomarkers: The study demonstrates that even a simple task of describing an image can reveal subtle linguistic changes associated with cognitive decline.
  • Semantic and lexical features are crucial: The most impactful features for discrimination were semantic (Word2Vec Distance) and N-gram features, highlighting that the content and structure of language are significantly affected by cognitive impairment.
  • Word2Vec Distance as a novel semantic feature: The proposed Word2Vec Distance feature, which measures how well a description aligns with the language used by healthy individuals to describe the same image, proved to be a powerful indicator of decline.
  • General pattern of linguistic deterioration: The findings support the hypothesis of a general pattern of linguistic deterioration in dementia, affecting lexical and semantic abilities while phonology and morphology remain relatively intact.
  • Misspellings as a potential indicator: An increased misspelling ratio was also found to be a contributing factor, suggesting a decline in basic lexical processing.

Key Results

  • The developed classifier achieved an accuracy of 86.1% in discriminating between subjects with normal aging and those with abnormal cognitive decline.
  • This performance significantly outperformed a baseline classifier (76.6% accuracy).
  • An ablation study revealed that semantic features (Word2Vec Distance) and N-gram features were the most important contributors to the classifier's performance.
  • The classifier correctly identified 24 out of 47 patients in the "Declined" group and 149 out of 154 normal patients.
  • The most significant errors occurred with subjects exhibiting mild impairments.

Limitations and Future Work

  • Label Uncertainty: The authors acknowledge that the "normal" labels in their corpus might not be perfectly stable, as some individuals labeled as normal may later show signs of decline, and MCI patients can sometimes recover. This inherent uncertainty in the ground truth can affect classification accuracy.
  • Small Corpus Size: While unique, the corpus of 201 descriptions is still relatively small, which can limit the generalizability of findings and the effectiveness of certain NLP techniques (e.g., word N-grams).
  • Future Work: The researchers plan to design a classifier based on Conditional Random Fields (CRFs) that will integrate a wider range of patient information (medical, cognitive, linguistic, and imaging) and track performance over time. They also intend to conduct further analysis of classification errors with specialized pathologists.

Summary updated

BibTeX
@inproceedings{weissenbacher2016automatic,
  author = {Davy Weissenbacher and Travis A. Johnson and Laura Wojtulewicz and Amylou Dueck and Dona Locke and Richard Caselli and Graciela Gonzalez},
  title = {{Automatic Prediction of Linguistic Decline in Writings of Subjects with Degenerative Dementia}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2016},
}
Figure