Isolating effects of age with fair representation learning when assessing dementia

Zining Zhu, Jekaterina Novikova, Frank Rudzicz

Abstract

One of the most prevalent symptoms among the elderly population, dementia, can be detected by classifiers trained on linguistic features extracted from narrative transcripts. However, these linguistic features are impacted in a similar but different fashion by the normal aging process. Aging is therefore a confounding factor, whose effects have been hard for machine learning classifiers (especially deep neural network based models) to ignore. We show DNN models are capable of estimating ages based on linguistic features. Predicting dementia based on this aging bias could lead to potentially non-generalizable accuracies on clinical datasets, if not properly deconfounded. In this paper, we propose to address this deconfounding problem with fair representation learning. We build neural network classifiers that learn low-dimensional representations reflecting the impacts of dementia yet discarding the effects of age. To evaluate these classifiers, we specify a model-agnostic score $\Delta_{eo}^{(N)}$ measuring how classifier results are deconfounded from age. Our best models compromise accuracy by only 2.56% and 1.54% on two clinical datasets compared to DNNs, and their $\Delta_{eo}^{(2)}$ scores are better than statistical (residulization and inverse probability weight) adjustments.

Introduction

Detecting dementia through speech analysis is a promising clinical tool. Dementia significantly impacts language, leading to frequent pauses, simplified syntax, and memory-related speech patterns. However, normal aging also alters speech in similar ways. In many clinical datasets, age and dementia are highly correlated—the impaired group is often significantly older than the control group.

Standard Deep Neural Networks (DNNs) are highly proficient at identifying a speaker's age from linguistic features. If a model learns that "older equals dementia," it may achieve high accuracy on a specific dataset by acting as an age-detector rather than a symptom-detector. This confounding problem limits generalization to younger patients with early-onset dementia or healthy older adults. Traditional statistical fixes like residualization often fail to capture the non-linear relationships inherent in deep learning, necessitating a more robust, "age-agnostic" approach.

Methods: Fair Representation Learning

The researchers propose using Fair Representation Learning via adversarial training to learn a low-dimensional representation ($z$) of speech features that preserves diagnostic information while discarding age information.

Proposed fair representation learning architectures: (a) simple adversarial, (b) autoencoder/entropy-based, and (c) consensus-net.

The study evaluates four model architectures:

  1. age-indep-simple: An "interpreter" creates a representation $z$. A "classifier" predicts dementia from $z$, while an "adversary" predicts age. The interpreter is trained to assist the classifier while "confusing" the adversary.
  2. age-indep-autoencoder: Incorporates a reconstruction task to ensure the representation $z$ retains general information about the original speech.
  3. age-indep-consensus-net: Splits features into modalities and encourages them to agree on an age-blind representation.
  4. age-indep-entropy: Explicitly maximizes the entropy (uncertainty) of the adversary’s age predictions.

To quantify success, the authors introduced $\Delta_{eo}^{(N)}$, an extension of "Equalized Odds" for continuous attributes like age. It measures the variance of classifier error rates across different age groups.

Datasets

The study utilized two primary datasets:

  • DementiaBank (DB): A public dataset of narrative picture descriptions (the "Cookie Theft" task) from subjects aged 45 to 90.
  • Famous People (FP): Transcripts from 17 public figures (e.g., Ronald Reagan, Gene Wilder), providing a wide range of ages and speech styles across the progression of dementia.

Key Insights and Results

The research yielded several critical findings:

  • DNNs as Age-Detectors: Baseline neural networks could predict a speaker's age within a mean absolute error of ~14-15 years from linguistic features alone, confirming that age bias is a default behavior.
  • Minimal Accuracy Trade-off: Fair models experienced only a slight dip in accuracy (1.5% to 2.5%) compared to biased DNNs, suggesting that age-agnostic features remain highly predictive.
  • Significant Bias Reduction: On DementiaBank, the best fair model reduced the bias metric ($\Delta_{eo}^{(2)}$) from 0.17 to 0.08.
  • Simplicity and Performance: Simpler adversarial models (age-indep-simple and age-indep-autoencoder) outperformed more complex ones. The authors hypothesize that extra parameters in complex models may inadvertently allow bias to leak.
  • Superiority over Traditional Statistics: Fair representation learning was more effective than Inverse Probability Weighting (IPW) or Residualization and does not require age data at test time.

Conclusion and Future Work

While effective, the $\Delta_{eo}^{(N)}$ metric's sensitivity to small sample sizes per age group remains a challenge in clinical datasets. Future work could extend this framework to other protected attributes, such as gender or education level, which also influence linguistic patterns. By isolating the effects of age, these models ensure that dementia detection is based on clinical symptoms rather than demographic shortcuts, improving the reliability and generalizability of medical AI.

Summary updated

Abstract

One of the most prevalent symptoms among the elderly population, dementia, can be detected by classifiers trained on linguistic features extracted from narrative transcripts. However, these linguistic features are impacted in a similar but different fashion by the normal aging process. Aging is therefore a confounding factor, whose effects have been hard for machine learning classifiers (especially deep neural network based models) to ignore. We show DNN models are capable of estimating ages based on linguistic features. Predicting dementia based on this aging bias could lead to potentially non-generalizable accuracies on clinical datasets, if not properly deconfounded. In this paper, we propose to address this deconfounding problem with fair representation learning. We build neural network classifiers that learn low-dimensional representations reflecting the impacts of dementia yet discarding the effects of age. To evaluate these classifiers, we specify a model-agnostic score $\Delta_{eo}^{(N)}$ measuring how classifier results are deconfounded from age. Our best models compromise accuracy by only 2.56\% and 1.54\% on two clinical datasets compared to DNNs, and their $\Delta_{eo}^{(2)}$ scores are better than statistical (residulization and inverse probability weight) adjustments.
BibTeX
@misc{zhu2018isolating,
  author = {Zining Zhu and Jekaterina Novikova and Frank Rudzicz},
  title = {{Isolating effects of age with fair representation learning when assessing dementia}},
  year = {2018},
  eprint = {1807.07217},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.07217},
}
Figure