An Adaptive Transfer-Learning-Based Deep Cox Neural Network for Hepatocellular Carcinoma Prognosis Prediction

Hua Chai, Xia Long, Lei Zhang, Jiarui Yang, Zhongyue Zhang, Xiangjun Qian, Yuedong Yang, Weidong PanFrontiers in Oncology 2021 · DOI

Abstract

These results proved that our proposed deep-learning-based method for HCC prognosis prediction is robust, accurate, and biologically meaningful.

Introduction

Hepatocellular Carcinoma (HCC) remains a leading cause of cancer-related mortality worldwide. Accurate prognosis prediction is essential for tailoring clinical interventions, such as determining the suitability for surgical resection versus targeted therapies. While high-throughput gene expression profiling provides a wealth of molecular data, its application in survival analysis is often hampered by the "small $n$, large $p$" problem—where the number of genetic features vastly exceeds the available patient samples.

To address this, researchers have turned to transfer learning, pre-training models on large pan-cancer datasets before fine-tuning them on specific targets like HCC. However, "negative transfer" can occur if the source and target cancers are biologically dissimilar. The challenge lies in objectively selecting the most relevant "donor" cancers to improve model performance.

Methodology: The ATRCN Framework

The authors developed the Adaptive Transfer-learning-based deep Cox neural network (ATRCN) to overcome these limitations. The workflow integrates deep learning with survival analysis and biological validation.

Overview of the ATRCN framework for liver cancer survival analysis and prognostic marker identification.

Cancer Characterization and Adaptive Selection

To ensure the model learns from the most relevant sources, the team represented 11 candidate cancer types from The Cancer Genome Atlas (TCGA) using a multi-dimensional description matrix. This matrix included:

  • 12 Phenotype Features: Survival metrics such as three-year survival rates and survival time quartiles.
  • 10 Genotype Features: Statistical summaries of mRNA expression data, compressed using Kernel Principal Component Analysis (KPCA).

Using k-means clustering on these features, the model adaptively identified pancreatic adenocarcinoma (PAAD) and stomach adenocarcinoma (STAD) as the most biologically similar "teachers" for HCC. This selection aligns with the shared gastrointestinal origins of these cancers.

Cancer characterization (A) and distance matrix (C) used for adaptive selection of pre-training datasets.

Deep Cox Modeling

The ATRCN architecture consists of a deep neural network with three hidden layers coupled with a Cox proportional hazards module. After pre-training on the selected donor cancers, the model is fine-tuned using HCC-specific data. This approach allows the network to leverage shared biological patterns while specializing in liver cancer prognosis.

Results and Validation

Predictive Performance

ATRCN demonstrated superior accuracy, achieving an average C-index of 0.628. This represents a 3.8% improvement over standard transfer learning and a 7.5% improvement over models trained without transfer learning (e.g., Deep_surv). The model's robustness was further confirmed on three independent Gene Expression Omnibus (GEO) datasets (GSE10143, GSE14520, and GSE54236), where it consistently separated patients into high- and low-risk groups ($p < 0.05$).

Discovery of TTC36

Beyond survival prediction, the study utilized Weighted Gene Co-expression Network Analysis (WGCNA) to identify key prognostic markers. Notably, the gene TTC36 was identified as a novel prognostic factor. In vitro experiments using Huh7 liver cancer cells revealed that overexpressing TTC36 significantly inhibited cell proliferation, migration, and invasion, validating its role as a tumor suppressor in HCC.

Conclusion and Future Directions

The ATRCN model provides a robust framework for cancer prognosis by intelligently selecting pre-training data. While the current study focused on mRNA expression, future work aims to incorporate multi-omics data—including DNA methylation and pathological imaging—to enhance predictive power. Additionally, exploring weighted transfer learning schemes could further refine how different donor cancers contribute to the model's prior knowledge.

Summary updated

Abstract

These results proved that our proposed deep-learning-based method for HCC prognosis prediction is robust, accurate, and biologically meaningful.
BibTeX
@inproceedings{chai2021adaptive,
  author = {Hua Chai and Xia Long and Lei Zhang and Jiarui Yang and Zhongyue Zhang and Xiangjun Qian and Yuedong Yang and Weidong Pan},
  title = {{An Adaptive Transfer-Learning-Based Deep Cox Neural Network for Hepatocellular Carcinoma Prognosis Prediction}},
  booktitle = {Frontiers in Oncology},
  year = {2021},
  doi = {10.3389/fonc.2021.692774},
}
Figure