Optimal diagnostic tests for sporadic Creutzfeldt-Jakob disease based on support vector machine classification of RT-QuIC data

William Hulme, Peter Richtárik, Lynne McGuire, Alison GreenarXiv 2012 · OpenReview

TL;DR

This paper explores using Support Vector Machines (SVMs) to improve diagnostic tests for Creutzfeldt-Jakob disease (sCJD) by analyzing real-time protein aggregation data from cerebrospinal fluid. The authors demonstrate that while the current best test relies on a single data point, SVMs can leverage more of the longitudinal data to achieve similar or better performance, particularly by focusing on the final fluorescence reading.

What Problem Does This Paper Address?

Current diagnostic tests for sporadic Creutzfeldt-Jakob disease (sCJD) rely on detecting markers of neuronal damage in cerebrospinal fluid (CSF). However, these markers are not specific to sCJD, leading to reduced specificity. A newer technique, Real-time Quaking-Induced Conversion (RT-QuIC), shows promise by detecting the aggregation of a specific protein implicated in sCJD. The existing RT-QuIC test uses a subjective threshold based on the final fluorescence reading after 90 hours. This paper aims to address the lack of mathematical rigor in determining this threshold and explore whether a more optimal test can be developed by using machine learning, specifically Support Vector Machines (SVMs), to analyze the full longitudinal RT-QuIC data. Additionally, the authors investigate if incorporating other patient data or analyzing earlier time points can improve the diagnostic capabilities.

Methods

The core methodology involves applying Support Vector Machine (SVM) classification to the RT-QuIC data. The authors explore various SVM formulations, including:

  • Linear SVMs: Used to find a hyperplane that best separates sCJD positive and negative cases in a multi-dimensional feature space.
  • Soft Margins: Employed when the data is not perfectly linearly separable, allowing for some misclassifications while penalizing them.
  • Feature Selection (L1-regularization): Used to identify the most important features (e.g., specific time points or derived measures) for classification, aiming to reduce model complexity and improve interpretability.
  • Cross-validation (Leave-One-Out): A technique used to assess the robustness and generalization ability of the trained models by iteratively removing one data point, retraining the model, and testing on the removed point.
  • Feature Engineering: Various methods were explored to create informative features from the raw fluorescence curves, including:
  • Using the maximum fluorescence reading at 90 hours.
  • Analyzing single fluorescence readings at different time points before 90 hours.
  • Extracting derivative information (first and second derivatives) from the smoothed fluorescence curves using finite differences.
  • Parametrizing the curves using piecewise linear regression.

Datasets

The study utilizes a dataset comprising:

  • 55 neuropathologically confirmed sCJD patients: Categorized into three subtypes: CJD-MM (30), CJD-MV (17), and CJD-VV (8).
  • 53 patients with alternative diagnoses: Initially suspected of having sCJD but later confirmed to have other conditions.

This dataset of 108 samples was used for training and evaluating the SVM models.

Key Insights and Ideas

  • The 90-hour fluorescence reading is a strong predictor: The study consistently finds that the maximum fluorescence reading at 90 hours is the most informative single feature for distinguishing between sCJD positive and negative cases.
  • Subjective thresholds can be optimized: SVMs can mathematically derive optimal thresholds from the data, potentially improving upon the subjective threshold used in the existing RT-QuIC test.
  • Additional patient data is not highly informative for SVM classification: Factors like sex and age do not appear to provide significant discriminatory power when used as features in SVM models for sCJD diagnosis.
  • sCJD subtypes are not distinguishable by RT-QuIC data alone: The analysis suggests that the RT-QuIC fluorescence data, even when combined with other patient information, is insufficient to differentiate between the different subtypes of sCJD.
  • Robustness vs. Fit trade-off: While SVMs can achieve a good "fit" (low error on training data), this can sometimes come at the cost of "robustness" (generalization to new data), indicating potential overfitting.

Key Results

  • Optimal test based on 90-hour reading: The most effective tests identified by the SVM analysis are based on the fluorescence readings at 90 hours.
  • SVM-optimized threshold: For the 90-hour reading, an SVM-derived threshold of approximately 10,699 rfu was found to be optimal, offering a slight improvement in sensitivity and specificity compared to the original 10,000 rfu threshold.
  • Robustness of 90-hour tests: Tests based on the 90-hour reading, particularly when considering the maximum fluorescence, demonstrated good robustness, with minimal changes in classification when individual samples were removed from the training set.
  • Limited utility of derivative features: While derivative features (rate of change, curvature) were explored, they did not offer significant improvements over using the maximum fluorescence reading, as they were highly correlated with it.
  • Inability to distinguish sCJD subtypes: SVM analysis failed to find any discernible differences in the RT-QuIC data that could reliably distinguish between CJD-MM, CJD-MV, and CJD-VV subtypes.

Limitations and Future Work

  • Non-linear kernels: The study primarily focused on linear SVMs. While non-linear kernels (Gaussian, polynomial) were investigated, issues with recovering primal solutions from dual solutions prevented a rigorous cross-validation analysis. The preliminary findings suggested they did not offer significant advantages.
  • Alternative classification algorithms: The paper did not explore other classification algorithms, such as logistic regression, which could potentially leverage probabilistic outputs and relationships between features (e.g., age and fluorescence) more effectively.
  • Crude smoothing techniques: The moving-average smoothing technique used is described as crude. More sophisticated methods like the Savitsky-Golay filter could potentially yield better derivative estimates.
  • Noise as information: The study assumes noise is solely due to equipment limitations and does not explore the possibility that noise itself might contain informative patterns.
  • Ignoring replicate relationships: The heuristic of selecting the replicate with the maximum fluorescence at 90 hours means that the relationships between replicates from the same sample were largely ignored, which could potentially hold discriminatory information.
  • Future work suggestions: The authors suggest exploring probabilistic classifiers, more advanced smoothing techniques, and investigating the potential information within noise or inter-replicate relationships.

Summary updated

Abstract

In this work we study numerical construction of optimal clinical diagnostic tests for detecting sporadic Creutzfeldt-Jakob disease (sCJD). A cerebrospinal fluid sample (CSF) from a suspected sCJD patient is subjected to a process which initiates the aggregation of a protein present only in cases of sCJD. This aggregation is indirectly observed in real-time at regular intervals, so that a longitudinal set of data is constructed that is then analysed for evidence of this aggregation. The best existing test is based solely on the final value of this set of data, which is compared against a threshold to conclude whether or not aggregation, and thus sCJD, is present. This test criterion was decided upon by analysing data from a total of 108 sCJD and non-sCJD samples, but this was done subjectively and there is no supporting mathematical analysis declaring this criterion to be exploiting the available data optimally. This paper addresses this deficiency, seeking to validate or improve the test primarily via support vector machine (SVM) classification. Besides this, we address a number of additional issues such as i) early stopping of the measurement process, ii) the possibility of detecting the particular type of sCJD and iii) the incorporation of additional patient data such as age, sex, disease duration and timing of CSF sampling into the construction of the test.
BibTeX
@misc{hulme2012optimal,
  author = {William Hulme and Peter Richtárik and Lynne McGuire and Alison Green},
  title = {{Optimal diagnostic tests for sporadic Creutzfeldt-Jakob disease based on support vector machine classification of RT-QuIC data}},
  year = {2012},
  eprint = {1212.2617},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1212.2617},
}
Figure