Bayesian inference of a new Mallows model for characterising symptom sequences applied in primary progressive aphasia

Beatrice Taylor, Cameron Shand, Chris J. D. Hardy, Neil OxtobyarXiv 2023 · OpenReview

TL;DR

This paper introduces a novel Bayesian Mallows model capable of handling partially ranked and right-censored symptom data, a common challenge in healthcare. Applied to primary progressive aphasia (PPA), the model successfully infers symptom orderings, offering a more nuanced understanding of disease progression than existing methods.

What Problem Does This Paper Address?

Understanding the sequence and co-occurrence of symptoms is crucial for effective healthcare, especially for complex, progressive conditions like dementia. Primary progressive aphasia (PPA), a rare language-focused dementia, presents with heterogeneous symptoms that accumulate over time. Existing disease progression models often rely on imaging data and struggle with the inherent messiness of real-world healthcare data, such as incomplete symptom reporting (partial rankings) and symptoms that haven't yet manifested (right-censored data). This paper aims to bridge this gap by developing a probabilistic model that can robustly characterize symptom sequences even with these data limitations.

Methods

The core of this work is an adaptation of the Mallows model, a probabilistic model for rankings. The authors extend it to handle:

  • Partial Rankings: Patients may not report on all symptoms, leading to incomplete orderings. The model uses a modified Kendall's Tau distance metric with a penalty parameter (p) to account for this.
  • Right-Censored Data: In healthcare surveys, patients might not have experienced certain symptoms yet. The authors address this by simply omitting comparisons involving unranked or unobserved symptoms from the distance calculation.

They employ a Bayesian inference framework with Markov Chain Monte Carlo (MCMC) methods to estimate the model parameters: the central ranking (π₀) and the spread parameter (λ), which indicates the consensus around the central ranking. They also derive a maximally likely ranking (MAP) estimate.

Datasets

  • Synthetic Data: Used to evaluate the model's performance under controlled conditions, mimicking the characteristics of real-world data (partial rankings, right-censoring).
  • Primary Progressive Aphasia (PPA) Dataset: A novel healthcare dataset consisting of symptom questionnaires from individuals with PPA and their caregivers. This dataset includes 72 symptoms, from which a subset of 8 well-being related symptoms was selected for modeling due to data sparsity.

Key Insights and Ideas

  • Mallows Model for Messy Healthcare Data: The adaptation of the Mallows model to handle partial and censored rankings is a significant step towards applying sophisticated probabilistic models to real-world clinical data.
  • Symptom Sequencing in PPA: The model successfully identified a plausible ordering of symptoms in PPA, aligning with clinical intuition and providing a data-driven way to understand disease progression.
  • Visualizing Symptom Co-occurrence: The model's output (central ordering) allows for a visual grouping of symptoms that tend to co-occur, offering a more intuitive understanding than simple frequency counts.
  • Scalability Challenges: The computational complexity of calculating the normalizing constant for the Mallows model, especially with partial rankings, poses a significant scalability challenge.

Key Results

  • Synthetic Data Performance: The model accurately inferred the true parameters used to generate synthetic data. Performance degraded gracefully with increasing percentages of missing data, as expected.
  • PPA Symptom Ordering: The model identified a central symptom ordering for PPA that was consistent with clinically informed rankings. For instance, "changes to sleeping patterns" was identified as an early symptom, and "difficulty swallowing" as a later one.
  • Comparison with EBM: The paper presents a comparison with the ordinal Event Based Model (EBM), showing that the Mallows model provides a more nuanced view by revealing symptom co-occurrence patterns, whereas EBM assigns a unique stage to each symptom.

Limitations and Future Work

  • Scalability: The primary limitation acknowledged is the computational scalability of the model, particularly the calculation of the normalizing constant. This hinders its application to very large datasets or a high number of symptoms.
  • Small Dataset Size: The PPA dataset was relatively small (30 individuals), which, combined with the complexity of the model, presents a challenge.
  • Future Work: The authors suggest exploring more accurate analytic approximations for the normalizing constant, rigorous ablation studies to test model performance under various conditions (e.g., censoring levels), and applying the model to other datasets like Activities of Daily Living questionnaires (ADLQ) or data from wearable technology. They also highlight the potential for alternative optimization methods for scenarios with weak consensus.

Summary updated

Abstract

Machine learning models offer the potential to understand diverse datasets in a data-driven way, powering insights into individual disease experiences and ensuring equitable healthcare. In this study, we explore Bayesian inference for characterising symptom sequences, and the associated modelling challenges. We adapted the Mallows model to account for partial rankings and right-censored data, employing custom MCMC fitting. Our evaluation, encompassing synthetic data and a primary progressive aphasia dataset, highlights the model's efficacy in revealing mean orderings and estimating ranking variance. This holds the potential to enhance clinical comprehension of symptom occurrence. However, our work encounters limitations concerning model scalability and small dataset sizes.
BibTeX
@misc{taylor2023bayesian,
  author = {Beatrice Taylor and Cameron Shand and Chris J. D. Hardy and Neil Oxtoby},
  title = {{Bayesian inference of a new Mallows model for characterising symptom sequences applied in primary progressive aphasia}},
  year = {2023},
  eprint = {2311.13411},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.13411},
}
Figure