Machine learning-based dynamic prediction of lateral lymph node metastasis in patients with papillary thyroid cancer

Sheng-wei Lai, Yunlong Fan, Yuhua Zhu, Fei Zhang, Zheng Guo, Bing Wang, Zheng Wan, Peilin Liu, Ning Yu, Handai QinFrontiers in Endocrinology 2022 · DOI

TL;DR

This paper introduces a machine learning-based web tool to predict lateral lymph node metastasis (LLNM) in papillary thyroid cancer (PTC) patients. The random forest model achieved the highest predictive accuracy, outperforming traditional nomograms, and the tool offers personalized risk assessment for clinicians.

What Problem Does This Paper Address?

Papillary thyroid cancer (PTC) is the most common type of thyroid cancer, and while generally having a good prognosis, lateral lymph node metastasis (LLNM) is a common complication. LLNM is an independent risk factor for cancer recurrence and can negatively impact disease-free survival. Current methods for detecting LLNM, such as ultrasound and fine-needle aspiration biopsy, have limitations in sensitivity, leading to potential delays in diagnosis and treatment. This leaves a critical need for accurate, objective tools to predict LLNM, especially in the preoperative setting, to guide surgical decisions and patient management.

Methods

The researchers employed a suite of six machine learning algorithms: decision tree, XGBoost, random forest, support vector machine, neural network, and K-nearest neighbor. They also compared these against a traditional logistic regression-based nomogram. Feature selection was a crucial step, utilizing LASSO regression to identify the most relevant 30 features from an initial set of 69. Model performance was evaluated using standard metrics like AUC, accuracy, precision, recall, F1 score, specificity, and sensitivity. To understand the model's decision-making process, they incorporated interpretable machine learning techniques, specifically SHAP (SHapley Additive exPlanations), to identify feature importance. Finally, they developed a web-based tool for practical clinical application.

Datasets

The study utilized a retrospective cohort of 1815 patients with papillary thyroid cancer who underwent thyroidectomy between January 2015 and December 2020 at the First Medical Centre of the Chinese PLA General Hospital. The dataset included clinical characteristics, laboratory findings, and ultrasound features. The data was split into an 80% training set and a 20% testing set for model development and validation. No new benchmark datasets were introduced; the focus was on applying existing data to develop and validate the predictive models.

Key Insights and Ideas

  • Random Forest Dominance: The random forest algorithm emerged as the top performer, demonstrating superior predictive accuracy for LLNM compared to other machine learning models and the traditional nomogram. This highlights its robustness in handling complex, non-linear relationships within the data.
  • Interpretability is Key: The use of SHAP values was instrumental in not only identifying the most influential predictors but also in building trust in the machine learning model. This addresses a common critique of "black box" AI in healthcare.
  • Web Tool for Accessibility: Creating a freely accessible web tool is a significant step towards translating research findings into practical clinical utility, allowing for individual-level risk assessment.
  • Key Predictors Identified: The study pinpointed several critical factors for LLNM prediction, including tumor size, lymph node microcalcification, age, lymph node size, and tumor location. This provides valuable insights for both clinicians and future research.

Key Results

The random forest model achieved an AUC of 0.80, with an accuracy of 0.74, sensitivity of 0.89, and an F1 score of 0.81. Decision curve analysis (DCA) indicated that the random forest model offered a higher clinical net benefit compared to the traditional nomogram and other machine learning models across a reasonable range of risk thresholds. The top five most influential features identified by SHAP analysis were tumor size, lymph node microcalcification, age, lymph node size, and tumor location. The web-based tool is available at http://43.138.62.202/.

Limitations and Future Work

The authors acknowledge several limitations:

  • Retrospective and Monocentric Design: The study's retrospective nature and reliance on data from a single medical center may limit the generalizability of the findings to broader populations and different healthcare settings.
  • Potential Omission of Variables: Despite feature selection, there's a possibility that some important variables might have been excluded, potentially impacting model performance.
  • Lack of External Validation: The study did not include external validation, meaning it's unclear how well the model would perform on independent datasets from different institutions.

Future work should focus on external validation to confirm the model's generalizability. Further research is also needed to clarify the optimal cut-off thresholds for key predictors like tumor size and age, and to explore the clinical implications and applications of the developed web tool more extensively. The authors also suggest that better algorithms will likely emerge for clinical disease prediction in the future.

Summary updated

Abstract

The results showed that machine learning can be used to enable accurate prediction for LLNM in PTC patients, and that the web tool allowed for LLNM risk assessment at the individual level.
BibTeX
@inproceedings{lai2022machine,
  author = {Sheng-wei Lai and Yunlong Fan and Yuhua Zhu and Fei Zhang and Zheng Guo and Bing Wang and Zheng Wan and Peilin Liu and Ning Yu and Handai Qin},
  title = {{Machine learning-based dynamic prediction of lateral lymph node metastasis in patients with papillary thyroid cancer}},
  booktitle = {Frontiers in Endocrinology},
  year = {2022},
  doi = {10.3389/fendo.2022.1019037},
}
Figure