Predicting the Long-Term Outcomes of Biologics in Psoriasis Patients Using Machine Learning

Seyedeh Sepideh Emam, Amy X. Du, Philip Surmanowicz, Simon F. Thomsen, Russ Greiner, Robert GniadeckiarXiv 2019 · OpenReview

TL;DR

This paper demonstrates that machine learning, specifically Generalized Linear Models (GLMs), can accurately predict the risk and duration of biologic treatment for psoriasis patients. By leveraging readily available clinical data, these models offer a promising tool for personalized treatment decisions and improved patient communication, achieving over 80% accuracy in predicting discontinuation.

What Problem Does This Paper Address?

Psoriasis is a chronic skin condition that, while significantly improved by biologic therapies, still presents challenges in long-term management. A major hurdle is that approximately 50% of patients discontinue these expensive treatments due to loss of efficacy or adverse events. Current decision-making for choosing the right biologic and predicting its success is largely trial-and-error, leading to suboptimal outcomes and wasted resources. While some predictors of discontinuation exist (e.g., obesity, female sex), their individual predictive power is low, necessitating a more sophisticated approach to personalize treatment.

Methods

The researchers employed a suite of seven supervised machine learning algorithms to predict biologic treatment outcomes in psoriasis patients. These included Generalized Linear Models (GLM), Logistic Regression, Deep Learning, Decision Tree, Random Forest, Gradient Boosted Trees, and Support Vector Machine. The primary goal was to identify the algorithm that best balanced accuracy, interpretability, and runtime. To evaluate performance, they utilized 5-fold cross-validation, a standard technique for robust model assessment and prevention of overfitting. Performance was measured using confusion matrices, accuracy, and Receiver Operating Characteristic (ROC) curves with Area Under the Curve (AUC) analysis. For predicting treatment length, a Bland-Altman analysis was used to compare predicted vs. actual durations.

Datasets

The study utilized a dataset from the Danish registry cohort, Dermbio, comprising 681 psoriasis patients who received biologic treatment between 2003 and 2013. This dataset contained anonymized individual patient data, including demographics, disease-specific characteristics, and therapy details. The paper does not introduce a new benchmark dataset.

Key Insights and Ideas

  • Machine learning can move beyond trial-and-error: The study shows that ML can predict treatment outcomes with high accuracy, moving away from the current "guess and check" approach for biologic selection.
  • GLMs are surprisingly effective: Despite the availability of more complex models like Deep Learning, the simpler Generalized Linear Model (GLM) emerged as a top performer, offering a good balance of accuracy and computational efficiency. This highlights the power of well-established statistical models when applied to relevant data.
  • Predicting why matters: The models not only predicted the risk of discontinuation but also its cause (lack of efficacy vs. adverse event), which is crucial for guiding subsequent treatment decisions.
  • Identifying the "ideal" patient: Input optimization revealed a profile of patients most likely to benefit from long-term biologic therapy, including younger, biologic-naive individuals with specific disease characteristics. This offers a tangible way to communicate prognosis.
  • Personalized prognosis is achievable: The ability to predict treatment length with a low mean absolute error suggests that patients can be given more concrete expectations about their therapy duration.

Key Results

  • High accuracy in predicting discontinuation: All tested ML algorithms achieved high accuracy, with the Generalized Linear Model (GLM) reaching 82% diagnostic accuracy for predicting drug discontinuation. The classification error was less than 18.46%.
  • Efficient prediction: The GLM model achieved this accuracy in under 2 seconds per patient, making it practical for clinical use.
  • Predicting treatment length: A separate GLM model predicted treatment length with a Mean Absolute Error (MAE) of 4.5 months, showing a strong linear dependency (Pearson Correlation Coefficient of 0.935) with actual treatment lengths.
  • Ideal patient profile: Patients with the best chance of long-term success were identified as: biologic-naive, under 49 years old at treatment initiation, early-onset plaque psoriasis (before age 23), no psoriatic arthritis, weight < 100 kg, and moderate-to-severe psoriasis activity (DLQI ≥ 16; PASI ≥ 10).
  • Decision Tree insights: While slightly less accurate than GLM, the Decision Tree analysis provided interpretable rules, suggesting that treatment duration significantly impacts discontinuation rates, with inefficacy being a primary driver for shorter treatments and adverse events for longer ones.

Limitations and Future Work

The authors acknowledge several limitations:

  • Small sample size and limited variables: The dataset, while containing valuable information, was relatively small and offered only 14 predictive variables. They suggest that incorporating more data (e.g., biologic dosage, detailed comorbidity profiles, concomitant medications) could further improve accuracy.
  • Retrospective nature: The study relied on retrospective data, which is common in ML but limits causal inference.
  • Single-country data: The dataset originated from a single country, meaning that local prescription practices, access to medication, and reimbursement policies might influence outcomes and generalizability.

Future work could involve:

  • Prospective studies: Conducting prospective studies to directly compare ML-guided treatment decisions with physician-led decisions.
  • Larger, multi-center datasets: Expanding the analysis to include data from multiple countries and larger patient cohorts.
  • Incorporating more variables: Integrating a wider range of clinical and patient-reported data to build more comprehensive predictive models.
  • Developing a personalized risk simulator: Translating these models into user-friendly tools for clinicians and patients to visualize expected outcomes.

Summary updated

Abstract

Background. Real-world data show that approximately 50% of psoriasis patients treated with a biologic agent will discontinue the drug because of loss of efficacy. History of previous therapy with another biologic, female sex and obesity were identified as predictors of drug discontinuations, but their individual predictive value is low. Objectives. To determine whether machine learning algorithms can produce models that can accurately predict outcomes of biologic therapy in psoriasis on individual patient level. Results. All tested machine learning algorithms could accurately predict the risk of drug discontinuation and its cause (e.g. lack of efficacy vs adverse event). The learned generalized linear model achieved diagnostic accuracy of 82%, requiring under 2 seconds per patient using the psoriasis patients dataset. Input optimization analysis established a profile of a patient who has best chances of long-term treatment success: biologic-naive patient under 49 years, early-onset plaque psoriasis without psoriatic arthritis, weight < 100 kg, and moderate-to-severe psoriasis activity (DLQI $\geq$ 16; PASI $\geq$ 10). Moreover, a different generalized linear model is used to predict the length of treatment for each patient with mean absolute error (MAE) of 4.5 months. However Pearson Correlation Coefficient indicates 0.935 linear dependencies between the actual treatment lengths and predicted ones. Conclusions. Machine learning algorithms predict the risk of drug discontinuation and treatment duration with accuracy exceeding 80%, based on a small set of predictive variables. This approach can be used as a decision-making tool, communicating expected outcomes to the patient, and development of evidence-based guidelines.
BibTeX
@misc{emam2019predicting,
  author = {Seyedeh Sepideh Emam and Amy X. Du and Philip Surmanowicz and Simon F. Thomsen and Russ Greiner and Robert Gniadecki},
  title = {{Predicting the Long-Term Outcomes of Biologics in Psoriasis Patients Using Machine Learning}},
  year = {2019},
  eprint = {1908.09251},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.09251},
}
Figure