Risk stratification of papillary thyroid cancers using multidimensional machine learning

Yuanhui Li, Fan Wu, Weigang Ge, Yu Zhang, Y. Hu, Lingqian Zhao, Wanglong Gou, Jingjing Shi, Yeqin Ni, Lu Li, Wenxin Fu, Xiangfeng Lin, Yunxian Yu, Zhijiang Han, Chuanghua Chen, Rujun Xu, Shirong Zhang, Li Zhou, Gang Pan, You Peng, Linlin Mao, Tianhan Zhou, Ju‐Sheng Zheng, Haitao Zheng, Yaoting Sun, Tiannan Guo, Dingcun LuoInternational Journal of Surgery 2023 · DOI

TL;DR

This paper introduces PRAC-PTC, a novel machine learning model that integrates clinical, genetic, immune, and proteomic data to accurately stratify papillary thyroid cancer (PTC) patients into low-risk and intermediate-/high-risk groups before surgery. This approach significantly improves upon existing methods and clinician assessments, potentially leading to more personalized treatment and reduced overtreatment.

What Problem Does This Paper Address?

Papillary thyroid cancer (PTC) is the most common endocrine malignancy, but its prognosis can vary significantly, with some cases having a high recurrence rate. Accurately assessing a patient's risk before surgery is crucial for determining the appropriate treatment strategy. Current methods, relying on clinical examinations, imaging, and sometimes genetic testing, often fall short in providing precise risk stratification. This can lead to either overtreatment (unnecessary surgery or aggressive therapies for indolent tumors) or undertreatment (inadequate management for aggressive tumors), both of which have negative consequences for patients and healthcare systems. The paper aims to bridge this gap by developing a more comprehensive and accurate preoperative risk assessment tool.

Methods

The core of this research is the development of a machine learning model, PRAC-PTC (Preoperative Risk Assessment Classifier for PTC). The authors employed a multi-dimensional approach, integrating data from four key areas:

  • Clinical Indicators: Standard patient information like age, tumor diameter, capsular invasion, extrathyroidal extension, and multifocality.
  • Immune Indices: Eight blood-based markers reflecting the body's systemic inflammatory response, such as lymphocyte-to-monocyte ratio (LMR) and systemic immune-inflammation index (SII).
  • Genetic Features: Specifically, the presence of the BRAFV600E mutation, a known factor in PTC prognosis.
  • High-Throughput Proteomics: This is a key novel aspect, utilizing mass spectrometry (specifically, PCT-DIA-MS) to analyze a vast array of proteins from tissue samples.

The machine learning model was built using the XGBoost algorithm, known for its robustness and performance in classification tasks. Feature selection was a critical step, identifying the most impactful variables from the combined dataset. The model's performance was rigorously validated on independent retrospective and prospective cohorts.

Datasets

The study utilized data from 558 PTC patients collected from two hospitals in China. These patients were divided into three cohorts:

  • Discovery Set (n=274): Used for initial model development and feature selection. Samples were formalin-fixed paraffin-embedded (FFPE).
  • Retrospective Test Set (n=166): Used for external validation of the developed model. Samples were also FFPE.
  • Prospective Test Set (n=118): Used for further external validation, utilizing fine-needle aspiration (FNA) biopsy samples.

No new benchmark datasets were introduced; rather, existing patient data from clinical settings were leveraged.

Key Insights and Ideas

  • Multidimensionality is Key: The study strongly suggests that no single data type is sufficient for accurate PTC risk stratification. Combining clinical, genetic, immune, and proteomic data yields significantly better predictive power than any single modality alone.
  • Proteomics as a Powerful Predictor: High-throughput proteomics, a relatively newer approach in this context, emerged as a crucial component, contributing significantly to the model's accuracy. The identification of specific proteins that differentiate risk levels is a major breakthrough.
  • Clinician Augmentation, Not Replacement: PRAC-PTC is designed to assist clinicians, not replace them. The model demonstrably improves the accuracy of risk assessment, particularly for junior clinicians, highlighting its potential as a decision-support tool.
  • Subtle but Important Genetic Role: While the BRAFV600E mutation is a known risk factor, its importance in the final PRAC-PTC model was relatively low compared to other features, suggesting a complex interplay of factors.

Key Results

  • High Predictive Accuracy: The PRAC-PTC model achieved an Area Under the Curve (AUC) of 0.925 in the discovery set.
  • Robust External Validation: The model was validated on independent cohorts, achieving AUCs of 0.787 for the retrospective FFPE test set and 0.799 for the prospective FNA test set.
  • Improved Clinician Performance: With PRAC-PTC assistance, the accuracy of clinicians improved significantly. For senior clinicians, accuracy rose from 59.8% (clinician alone) to 84.4% (retrospective test set) and 83.5% (prospective test set). Even junior clinicians saw substantial improvements in sensitivity and specificity.
  • Key Features Identified: The model identified 17 key features, including five clinical indicators, five immune indices, six proteins (e.g., DPP7, PDLIM3, Col12A1, CTSL, TUBB2A, ITGB5), and the BRAFV600E mutation. Notably, extrathyroidal extension and capsular invasion were identified as the most important clinical features.

Limitations and Future Work

The authors acknowledge several limitations:

  • Geographic Bias: The study primarily included Asian patients, and the model's generalizability to other ethnic populations needs further validation.
  • Sample Size and Centers: Expanding the sample size and including data from more diverse clinical centers are recommended to improve the model's robustness.
  • Proteomics Method: While powerful, the current proteomic analysis relies on FFPE or FNA samples, which can be invasive. Future work may explore less invasive sample sources like body fluids.
  • Targeted Proteomics: The study used untargeted proteomics. Future research could employ faster, more stable targeted proteomics or antibody-based methods for validation.
  • Further Subdivision: The current model stratifies into low-risk and intermediate-/high-risk. Further subdivision within the intermediate/high-risk categories could refine treatment decisions.
  • Model Performance: While good, the authors note that the predictive performance is "not ideal," suggesting room for improvement through multiomics sequencing and multimodal data integration.

Future work will focus on addressing these limitations, exploring other PTC subtypes, and refining the model for broader clinical application.

Summary updated

Abstract

This study demonstrated that the PRAC-PTC that integrating clinical data, gene mutation information, immune indices, high-throughput proteomics and machine learning technology in multicentre retrospective and prospective clinical cohorts can effectively stratify the preoperative risk of PTC and may decrease unnecessary surgery or overtreatment.
BibTeX
@article{li2023risk,
  author = {Yuanhui Li and Fan Wu and Weigang Ge and Yu Zhang and Y. Hu and Lingqian Zhao and Wanglong Gou and Jingjing Shi and Yeqin Ni and Lu Li and Wenxin Fu and Xiangfeng Lin and Yunxian Yu and Zhijiang Han and Chuanghua Chen and Rujun Xu and Shirong Zhang and Li Zhou and Gang Pan and You Peng and Linlin Mao and Tianhan Zhou and Ju‐Sheng Zheng and Haitao Zheng and Yaoting Sun and Tiannan Guo and Dingcun Luo},
  title = {{Risk stratification of papillary thyroid cancers using multidimensional machine learning}},
  journal = {International Journal of Surgery},
  year = {2023},
  doi = {10.1097/js9.0000000000000814},
}
Figure