Machine Learning-Assisted Recurrence Prediction for Early-Stage Non-Small-Cell Lung Cancer Patients

Adrianna Janik, Maria Torrente, Luca Costabello, Virginia Calvo, Brian Walsh, Carlos Camps, Sameh K. Mohamed, Ana L. Ortega, Vít Novácek, Bartomeu Massutí, Pasquale Minervini, M. Rosario Garcia Campelo, Edel del Barco, Joaquim Bosch-Barrera, Ernestina Menasalvas, Mohan Timilsina, Mariano ProvencioarXiv 2022 · OpenReview

TL;DR

This paper introduces machine learning models, both tabular and graph-based, to predict the recurrence of early-stage Non-Small-Cell Lung Cancer (NSCLC) in patients. The models achieve promising accuracy (up to 76%) and provide explainable predictions, aiming to personalize patient care and guide adjuvant treatment decisions.

What Problem Does This Paper Address?

Lung cancer remains a leading cause of cancer-related death globally. For early-stage NSCLC patients, surgical resection offers the best chance of a cure, but relapse rates remain high (30-55% even after curative surgery). Accurately identifying which patients are at higher risk of relapse is crucial for personalizing follow-up care and deciding on adjuvant therapies. Current prognostic indicators, like the TNM staging system, are limited. This paper addresses the need for more precise and personalized relapse prediction by leveraging machine learning.

Methods

The researchers employed two main machine learning approaches:

  • Tabular Machine Learning: They trained models (Gradient Boosting, Logistic Regression, and Random Forest) on patient data structured as tables. For explaining predictions, they used SHAP (SHapley Additive exPlanations) to show how each feature contributes to the predicted relapse probability.
  • Graph Machine Learning: They represented patient data as a knowledge graph, allowing for the modeling of complex relationships between various clinical entities. The ComplEx-N3 model was used for this approach. Explanations were generated using an example-based method, highlighting similar past patients from the training set.

Both approaches involved feature selection and hyperparameter tuning using grid search. The models were trained and evaluated using cross-validation and held-out test sets.

Datasets

The study utilized data from 1,387 early-stage (I-II) Non-Small-Cell Lung Cancer (NSCLC) patients collected by the Spanish Lung Cancer Group's Thoracic Tumor Registry (TTR). This dataset included a wide range of patient and treatment characteristics. No new benchmarks were introduced; the focus was on applying existing ML techniques to this specific clinical problem.

Key Insights and Ideas

  • Graph-based models can capture complex relationships: Representing patient data as a knowledge graph allows for the inference of long-range dependencies, potentially leading to more nuanced predictions than traditional tabular methods.
  • Explainability is crucial for clinical adoption: Providing explanations for ML predictions (e.g., via SHAP or example-based methods) is essential for building trust and enabling clinicians to make informed decisions.
  • Diverse features contribute to relapse risk: The study highlights that a combination of diagnosis, surgery, and treatment features are important for accurate relapse prediction.
  • Personalized risk stratification is achievable: The goal is to move beyond general risk categories to provide individualized relapse probabilities, paving the way for tailored treatment strategies.

Key Results

  • Tabular models achieved up to 76% accuracy: The Random Forest model, trained on tabular data, demonstrated a 76% accuracy in predicting relapse using 10-fold cross-validation.
  • Graph machine learning achieved 68% accuracy: The ComplEx-N3 graph model achieved 68% accuracy on a held-out test set.
  • Feature importance varies: SHAP analysis revealed that factors like family history of cancer, chemotherapy regimen, and surgery details significantly influenced relapse predictions.
  • Example-based explanations provide context: For graph models, highlighting similar past patients helps clinicians understand the basis of a prediction.

Limitations and Future Work

  • Geographic Bias: The study is limited to patients treated in Spain, which may affect the generalizability of the models to other populations with different healthcare practices.
  • Time-Awareness: The current models are time-unaware. Future work will focus on incorporating time-based features and developing time-aware models for more precise relapse prediction.
  • Data Collection Control: The researchers had no control over the original data collection process, which could introduce inconsistencies.
  • Future Data Integration: The authors suggest that incorporating patient-level molecular data and medical imaging (CT scans) could further enhance the prognostic model's accuracy and utility.
  • Prospective Validation: Further prospective and multi-site validation is needed to establish the model as a reliable decision support tool.

Summary updated

Abstract

Background: Stratifying cancer patients according to risk of relapse can personalize their care. In this work, we provide an answer to the following research question: How to utilize machine learning to estimate probability of relapse in early-stage non-small-cell lung cancer patients? Methods: For predicting relapse in 1,387 early-stage (I-II), non-small-cell lung cancer (NSCLC) patients from the Spanish Lung Cancer Group data (65.7 average age, 24.8% females, 75.2% males) we train tabular and graph machine learning models. We generate automatic explanations for the predictions of such models. For models trained on tabular data, we adopt SHAP local explanations to gauge how each patient feature contributes to the predicted outcome. We explain graph machine learning predictions with an example-based method that highlights influential past patients. Results: Machine learning models trained on tabular data exhibit a 76% accuracy for the Random Forest model at predicting relapse evaluated with a 10-fold cross-validation (model was trained 10 times with different independent sets of patients in test, train and validation sets, the reported metrics are averaged over these 10 test sets). Graph machine learning reaches 68% accuracy over a 200-patient, held-out test set, calibrated on a held-out set of 100 patients. Conclusions: Our results show that machine learning models trained on tabular and graph data can enable objective, personalised and reproducible prediction of relapse and therefore, disease outcome in patients with early-stage NSCLC. With further prospective and multisite validation, and additional radiological and molecular data, this prognostic model could potentially serve as a predictive decision support tool for deciding the use of adjuvant treatments in early-stage lung cancer. Keywords: Non-Small-Cell Lung Cancer, Tumor Recurrence Prediction, Machine Learning
BibTeX
@misc{janik2022machine,
  author = {Adrianna Janik and Maria Torrente and Luca Costabello and Virginia Calvo and Brian Walsh and Carlos Camps and Sameh K. Mohamed and Ana L. Ortega and Vít Novácek and Bartomeu Massutí and Pasquale Minervini and M. Rosario Garcia Campelo and Edel del Barco and Joaquim Bosch-Barrera and Ernestina Menasalvas and Mohan Timilsina and Mariano Provencio},
  title = {{Machine Learning-Assisted Recurrence Prediction for Early-Stage Non-Small-Cell Lung Cancer Patients}},
  year = {2022},
  eprint = {2211.09856},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2211.09856},
}
Figure