Two-stage biologically interpretable neural-network models for liver cancer prognosis prediction using histopathology and transcriptomic data

Zhucheng Zhan, Zheng Jing, Bing He, Noshad Hosseini, Maria Westerhoff, Eun-Young Choi, Lana X. GarmiremedRxiv 2020 · DOI

Introduction

Hepatocellular carcinoma (HCC) is the most prevalent form of liver cancer and remains a significant clinical challenge due to its high heterogeneity and poor prognosis, with a five-year survival rate of approximately 12%. Accurate prognostic prediction is essential for personalized clinical management. While histopathology slides and transcriptomic (RNA-Seq) data provide critical insights into tumor behavior, integrating these heterogeneous data types into a single predictive model is computationally demanding.

Traditional prognostic models often rely on single-modality data or utilize complex "black-box" deep learning architectures that are prone to overfitting and lack biological interpretability. This research introduces a two-stage biologically interpretable neural-network framework, based on Cox-nnet, designed to integrate histopathology and transcriptomic data for improved survival prediction in HCC patients.

Methodology: The Two-Stage Cox-nnet Framework

The proposed framework moves away from raw pixel-based analysis, which often requires massive datasets to avoid overfitting, and instead utilizes quantitative biological features. The workflow is divided into two distinct stages:

Stage 1: Feature Extraction and Modality-Specific Modeling

First, the researchers used CellProfiler to extract 2,429 quantitative features from TCGA histopathology images, including cell shape, texture, and staining intensity. Simultaneously, RNA-Seq data was processed to capture gene expression profiles. Two separate Cox-nnet models—a neural network extension of the Cox proportional hazards model—were trained in parallel: one for the imaging features and one for the transcriptomic data.

Stage 2: Integrative Meta-Modeling

The "hidden" representations (the internal patterns learned by the network) from both Stage 1 models were extracted and combined. These combined features served as the input for a second-stage "meta" Cox-nnet model. This hierarchical approach allows the model to learn complex interactions between visual tissue characteristics and underlying molecular signatures.

Experimental Results

The model was evaluated using data from The Cancer Genome Atlas (TCGA), specifically a cohort of 290 patients with matched high-quality imaging and genomic data. Performance was measured using the C-index (where 1.0 represents perfect prediction) and log-rank p-values.

Superior Predictive Accuracy

The two-stage integrative model achieved a median C-index of 0.75, significantly outperforming models based on a single data type:

  • Integrated Model (Images + Genes): 0.75
  • Image-only Model: 0.74
  • Gene-only Model: 0.70

Benchmarking Against State-of-the-Art

The framework was compared against PAGE-Net, a complex convolutional neural network (CNN) that integrates raw pixels with genomic data. The two-stage Cox-nnet significantly outperformed PAGE-Net (C-index 0.68). Furthermore, while PAGE-Net required over a week of training on high-end GPUs and was susceptible to overfitting, the feature-based Cox-nnet approach was computationally efficient and more stable.

Statistical Significance

The model successfully stratified patients into high-risk and low-risk groups with a highly significant log-rank p-value of 6e-7, demonstrating a much clearer separation than traditional statistical methods like Cox-PH or Random Survival Forests.

Biological Interpretation and Insights

A key advantage of this framework is its interpretability. The study found that imaging texture features were the most predictive visual indicators of prognosis. Interestingly, these top-performing image features showed modest but significant correlations with specific genes, such as MMP7 and KRT17. These genes are known to be involved in liver tissue morphogenesis and cancer progression, suggesting that the model is capturing a "biological handshake" between macroscopic tissue architecture and microscopic molecular activity.

Conclusion

The two-stage Cox-nnet framework provides a robust, interpretable method for multi-modal data integration in oncology. By combining the strengths of histopathology and transcriptomics, the model achieves higher accuracy than single-modality approaches or more complex, pixel-based deep learning models. This research highlights the potential of using pre-extracted biological features to build predictive tools that are both clinically relevant and biologically grounded. Future work will focus on expanding this framework to other cancer types and incorporating additional "omics" layers to further refine survival predictions.

Summary updated

Abstract

ABSTRACT Purpose Pathological images are easily accessible data with the potential as prognostic biomarkers. Moreover, integration of heterogeneous data types from multi-modality, such as pathological image and gene expression data, is invaluable to help predicting cancer patient survival. However, the analytical challenges are significant. Experimental Design Here we take the hepatocellular carcinoma (HCC) pathological image features extracted by CellProfiler, and apply them as the input for Cox-nnet, a neural network-based prognosis. We compare this model with conventional Cox-PH model, CoxBoost, Random Survival Forests and DeepSurv, using C-index and log ranked p-values on HCC testing samples. Further, to integrate pathological image and gene expression data of the same patients, we innovatively construct a two-stage Cox-nnet model, and compare it with another complex neural network model PAGE-Net. Results pathological image based prognosis prediction using Cox-nnet is significantly more accurate than Cox-PH and random survival forests models, and comparable with CoxBoost and DeepSurv methods. Additionally, the two-stage Cox-nnet complex model combining histopathology image and transcriptomics RNA-Seq data achieves better prognosis prediction, with a median C-index of 0.75 and log-rank p-value of 6e-7 in the testing datasets. The results are much more accurate than PAGE-Net, a CNN based complex model (median C-index of 0.68 and log-rank p-value of 0.03). Imaging features present additional predictive information to gene expression features, as the combined model is much more accurate than the model with gene expression alone (median C-index 0.70). Pathological image features are modestly correlated with gene expression. Genes having correlations to top imaging features have known associations with HCC patient survival and morphogenesis of liver tissue. Conclusion This work provides two-stage Cox-nnet, a new class of biologically relevant and relatively interpretable models, to integrate multi-modal and multiple types of data for survival prediction.
BibTeX
@misc{zhan2020two,
  author = {Zhucheng Zhan and Zheng Jing and Bing He and Noshad Hosseini and Maria Westerhoff and Eun-Young Choi and Lana X. Garmire},
  title = {{Two-stage biologically interpretable neural-network models for liver cancer prognosis prediction using histopathology and transcriptomic data}},
  year = {2020},
  doi = {10.1101/2020.01.25.20016832},
  howpublished = {medRxiv},
}
Figure