Feature Selection on Lyme Disease Patient Survey Data

Joshua Vendrow, Jamie Haddock, Deanna Needell, Lorraine Johnson

Abstract

Lyme disease is a rapidly growing illness that remains poorly understood within the medical community. Critical questions about when and why patients respond to treatment or stay ill, what kinds of treatments are effective, and even how to properly diagnose the disease remain largely unanswered. We investigate these questions by applying machine learning techniques to a large scale Lyme disease patient registry, MyLymeData, developed by the nonprofit LymeDisease.org. We apply various machine learning methods in order to measure the effect of individual features in predicting participants' answers to the Global Rating of Change (GROC) survey questions that assess the self-reported degree to which their condition improved, worsened, or remained unchanged following antibiotic treatment. We use basic linear regression, support vector machines, neural networks, entropy-based decision tree models, and $k$-nearest neighbors approaches. We first analyze the general performance of the model and then identify the most important features for predicting participant answers to GROC. After we identify the "key" features, we separate them from the dataset and demonstrate the effectiveness of these features at identifying GROC. In doing so, we highlight possible directions for future study both mathematically and clinically.

Introduction

Lyme disease remains a significant clinical enigma. While many patients recover after initial treatment, an estimated 36% suffer from persistent, debilitating symptoms. The medical community currently lacks a consensus on why some patients respond to treatment while others do not. Furthermore, the high-dimensional nature of patient survey data makes it difficult for researchers to identify meaningful patterns.

This study aims to bridge this gap by using machine learning to identify the "signal" within the "noise" of the MyLymeData registry. Specifically, the research focuses on determining which patient attributes, symptoms, and treatment protocols are most strongly associated with a patient’s Global Rating of Change (GROC)—a validated measure of whether a patient feels better, worse, or the same after treatment.

Methods

The researchers employed a "multi-model ensemble" approach to feature selection, testing five different models to ensure robust findings: Linear Regression, Support Vector Machines (SVM), Neural Networks (two dense layers with 20 nodes each), Entropy-based Decision Trees, and k-Nearest Neighbors (KNN).

To handle the inherent "messiness" of survey data, subsampling was used to create a balanced dataset (SMLD), preventing models from simply learning to predict the most common outcome. Features were then ranked by their predictive power across all five models. These rankings were aggregated to identify the "Top 30" most influential variables.

The study utilized data from Phase 1 of the MyLymeData patient registry, involving 2,162 patients with persistent Lyme disease. The dataset included 215 variables derived from survey questions covering health history, diagnosis, symptoms, and treatment. Specific subsets were created for testing: TSMLD (Top Subsampled MyLymeData, containing only the top 30 features) and BSMLD (Bottom Subsampled MyLymeData, containing the remaining 185 features).

Key Insights and Feature Importance

One of the most striking findings was that models trained on only the top 30 features often outperformed those trained on the full 215-feature set. This suggests that much of the survey data is redundant or introduces noise.

Feature importance ranking based on accuracy, highlighting fatigue (Sx_Sev_1) and antibiotic effectiveness (Abx_Eff) as top predictors.

As shown in the feature importance analysis, fatigue (Sx_Sev_1) emerged as the single most predictive symptom for overall improvement. Patients reporting minimal or no fatigue were significantly more likely to be "high responders" to treatment. Additionally, 20 of the top 30 features were directly related to antibiotic use (e.g., effectiveness, duration, and delivery methods), providing quantitative evidence for the central role of antibiotic protocols in patient outcomes.

Results

On the balanced 3-class problem (Non-responder, Low responder, High responder), the models achieved a peak accuracy of approximately 60% on the full dataset. Notably, using only the top 30 features (TSMLD) in the SVM model resulted in 54.6% accuracy, compared to only 48.4% when using the full 215-feature set.

Predictive accuracy of various machine learning models across different responder categories.

The study confirmed that the Global Rating of Change is a highly effective metric for segmenting patient populations in a way that correlates with specific clinical treatments. The results demonstrate that a small subset of features can capture the majority of the predictive signal.

Discussion and Future Work

While the study provides valuable insights, it is limited by the subjective nature of self-reported survey data, which is prone to recall bias. Additionally, computational constraints prevented the use of Neural Networks for the iterative individual feature selection process.

Future research could focus on optimizing patient surveys by concentrating on these 30 key features. This would reduce "survey fatigue" for patients while maintaining high data quality for researchers. Furthermore, while the study identifies features that predict improvement, further clinical investigation is needed to establish causal relationships between these factors and patient recovery.

Summary updated

Abstract

Lyme disease is a rapidly growing illness that remains poorly understood within the medical community. Critical questions about when and why patients respond to treatment or stay ill, what kinds of treatments are effective, and even how to properly diagnose the disease remain largely unanswered. We investigate these questions by applying machine learning techniques to a large scale Lyme disease patient registry, MyLymeData, developed by the nonprofit LymeDisease.org. We apply various machine learning methods in order to measure the effect of individual features in predicting participants' answers to the Global Rating of Change (GROC) survey questions that assess the self-reported degree to which their condition improved, worsened, or remained unchanged following antibiotic treatment. We use basic linear regression, support vector machines, neural networks, entropy-based decision tree models, and $k$-nearest neighbors approaches. We first analyze the general performance of the model and then identify the most important features for predicting participant answers to GROC. After we identify the "key" features, we separate them from the dataset and demonstrate the effectiveness of these features at identifying GROC. In doing so, we highlight possible directions for future study both mathematically and clinically.
BibTeX
@article{vendrow2020feature,
  author = {Joshua Vendrow and Jamie Haddock and Deanna Needell and Lorraine Johnson},
  title = {{Feature Selection on Lyme Disease Patient Survey Data}},
  journal = {Algorithms},
  year = {2020},
  doi = {10.3390/a13120334},
}
Figure