Predicting tumor recurrence on baseline MR imaging in patients with early-stage hepatocellular carcinoma using deep machine learning

Ahmet S. Kücükkaya, Tal Zeevi, Nathan Chai, Rajiv Raju, Stefan P. Haider, Mohamed Elbanan, Alexandra Petukhova-Greenstein, MingDe Lin, John A. Onofrey, Michal Nowak, Kirsten Cooper, Elizabeth Thomas, Jessica Gois Santana, Bernhard Gebauer, David C. Mulligan, Lawrence H. Staib, Ramesh Batra, Julius ChapiroScientific 2023 · DOI

TL;DR

This study leverages deep learning, specifically Convolutional Neural Networks (CNNs) and XGBoost, to predict the recurrence of early-stage hepatocellular carcinoma (HCC) using only baseline MR imaging. The models achieved moderate to high accuracy (AUC 0.71-0.85) and successfully stratified patients into high- and low-risk groups, paving the way for personalized follow-up strategies.

What Problem Does This Paper Address?

Hepatocellular carcinoma (HCC) is a deadly cancer, and even with early-stage treatment, recurrence rates are alarmingly high, reaching up to 70%. Current methods for predicting recurrence, like the Barcelona Clinic Liver Cancer (BCLC) system, rely on basic tumor characteristics (size, number) which fail to capture the underlying tumor biology or accurately estimate post-treatment risk. This limitation hinders the optimization of follow-up imaging and personalized treatment strategies. The paper aims to bridge this gap by exploring whether advanced deep learning techniques can extract subtle imaging features from pre-treatment MR scans to predict recurrence risk more effectively.

Methods

The researchers employed a machine learning pipeline that combined a pre-trained CNN (VGG16) for feature extraction with a gradient boosting classifier (XGBoost) for prediction.

  • Feature Extraction: A VGG16 CNN, pre-trained on the ImageNet dataset, was used to automatically extract relevant imaging features from the MR scans. This approach bypasses the need for manual feature engineering, which can be biased and may miss crucial information. The entire liver volume was considered, not just the tumor itself, to potentially capture the impact of the surrounding tissue and underlying liver conditions.
  • Prediction: The extracted features were then fed into an XGBoost model, a powerful ensemble learning algorithm known for its robustness.
  • Validation: A nested cross-validation strategy was used to ensure the model's reliability and prevent overfitting. This involved an outer loop for model testing (leave-one-out CV) and an inner loop for hyperparameter optimization (10-fold stratified Monte Carlo CV).
  • Clinical Relevance: To assess the practical utility of the predictions, Kaplan-Meier analysis was performed. Patients were stratified into "high-risk" and "low-risk" recurrence groups based on the model's output, and their recurrence-free survival (RFS) was analyzed.

Datasets

The study utilized a retrospective dataset of 120 adult patients diagnosed with early-stage HCC between 2005 and 2018 from a single institution. These patients had undergone surgical resection, thermal ablation, or liver transplantation as their first-line treatment and had achieved complete response post-treatment. The dataset included multi-parametric, contrast-enhanced pre-treatment MR imaging. No new benchmarks were introduced; the focus was on applying existing deep learning techniques to a clinical problem.

Key Insights and Ideas

  • Deep Learning's Power in Imaging: The study strongly suggests that deep learning, particularly CNNs, can automatically identify complex imaging patterns in MR scans that are predictive of HCC recurrence, going beyond what traditional staging systems can offer.
  • Whole Liver Input Matters: Using the entire liver volume as input for the CNN, rather than just segmented tumor regions, appears to be a crucial factor. This approach likely captures the influence of the background liver parenchyma and underlying cirrhotic changes, which are known to contribute to de novo recurrence.
  • Early Prediction for Personalized Care: The ability to predict recurrence risk at various time points (1-6 years) is a significant step towards tailoring follow-up schedules and potentially intervening earlier for high-risk patients.
  • MR Imaging as a Predictive Tool: The study highlights the potential of MR imaging, with its superior soft-tissue contrast compared to CT, as a valuable tool for pre-treatment risk stratification in HCC.

Key Results

  • Predictive Accuracy: Six different models were developed to predict recurrence within six distinct time frames (1-6 years). The Area Under the Receiver Operating Characteristic Curve (AUC-ROC) for these models ranged from 0.71 to 0.85, indicating moderate to high predictive performance. The average test-AUC across all models was 0.76 with a standard deviation of 0.05.
  • Clinical Stratification: Kaplan-Meier analysis demonstrated that the models could significantly stratify patients into "high-risk" and "low-risk" recurrence groups. This stratification achieved statistical significance (log-rank p < 0.05) for five out of the six time frames (2-6 years), suggesting a strong correlation between the model's predictions and actual recurrence-free survival. The 1-year prediction showed a trend towards significance (p=0.06).
  • Patient Characteristics: Of the 120 patients, 44 experienced disease recurrence. No statistically significant differences were found in baseline clinical characteristics between the recurrent and non-recurrent groups, underscoring the limitations of traditional assessment methods.

Limitations and Future Work

  • Retrospective and Single-Center Study: The study's retrospective nature and reliance on data from a single institution are acknowledged limitations. This means the findings may not be generalizable to other populations or healthcare settings.
  • Small Cohort Size: While 120 patients is a reasonable starting point, the authors note that the cohort size is relatively small, especially for predicting recurrence in later time frames. This can lead to a large data imbalance and potentially affect model robustness.
  • External Validation Needed: The lack of external validation is a significant limitation. The authors emphasize the need for external validation on larger, multi-center datasets to confirm the model's generalizability and reliability.
  • Limited 1-Year Prediction: The model's predictive power was not statistically significant for the 1-year recurrence prediction, which the authors attribute to the small number of recurrent cases and the data imbalance within that specific timeframe.
  • Future Directions: The authors propose several avenues for future research:
  • External Validation: Crucial for confirming the model's performance across different cohorts.
  • Incorporating Clinical Data: Combining imaging features with clinical parameters (e.g., gender, age, AFP, NLR, PLR) could potentially improve prediction accuracy through ensemble methods.
  • Multi-center Prospective Studies: To enhance reliability and clinical applicability.
  • Precision Medicine: The ultimate goal is to use these algorithms to individualize treatment and follow-up strategies for HCC patients.

Summary updated

Abstract

Tumor recurrence affects up to 70% of early-stage hepatocellular carcinoma (HCC) patients, depending on treatment option. Deep learning algorithms allow in-depth exploration of imaging data to discover imaging features that may be predictive of recurrence. This study explored the use of convolutional neural networks (CNN) to predict HCC recurrence in patients with early-stage HCC from pre-treatment magnetic resonance (MR) images. This retrospective study included 120 patients with early-stage HCC. Pre-treatment MR images were fed into a machine learning pipeline (VGG16 and XGBoost) to predict recurrence within six different time frames (range 1-6 years). Model performance was evaluated with the area under the receiver operating characteristic curves (AUC-ROC). After prediction, the model's clinical relevance was evaluated using Kaplan-Meier analysis with recurrence-free survival (RFS) as the endpoint. Of 120 patients, 44 had disease recurrence after therapy. Six different models performed with AUC values between 0.71 to 0.85. In Kaplan-Meier analysis, five of six models obtained statistical significance when predicting RFS (log-rank p < 0.05). Our proof-of-concept study indicates that deep learning algorithms can be utilized to predict early-stage HCC recurrence. Successful identification of high-risk recurrence candidates may help optimize follow-up imaging and improve long-term outcomes post-treatment.
BibTeX
@inproceedings{kucukkaya2023predicting,
  author = {Ahmet S. Kücükkaya and Tal Zeevi and Nathan Chai and Rajiv Raju and Stefan P. Haider and Mohamed Elbanan and Alexandra Petukhova-Greenstein and MingDe Lin and John A. Onofrey and Michal Nowak and Kirsten Cooper and Elizabeth Thomas and Jessica Gois Santana and Bernhard Gebauer and David C. Mulligan and Lawrence H. Staib and Ramesh Batra and Julius Chapiro},
  title = {{Predicting tumor recurrence on baseline MR imaging in patients with early-stage hepatocellular carcinoma using deep machine learning}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-34439-7},
}
Figure