Prototype-Guided Cross-Modal Knowledge Enhancement for Adaptive Survival Prediction

Fengchun Liu, Linghan Cai, Zhikang Wang, Zhiyuan Fan, Jin-gang Yu, Hao Chen, Yongbing Zhang

Abstract

Survival analysis is a critical component of precision medicine, increasingly relying on the integration of histopathological Whole-Slide Images (WSIs) and genomic profiles. However, the practical application of multimodal models is often hindered by the "missing modality" problem, where one data type is unavailable in clinical settings. This paper presents ProSurv, a novel framework designed for adaptive survival prediction. By utilizing modality-specific prototype banks and a cross-modal translation module, ProSurv effectively enhances unimodal features with synthetic knowledge from the missing modality. Experimental results across multiple TCGA datasets demonstrate that ProSurv achieves state-of-the-art performance in both multimodal and unimodal scenarios, offering a robust solution for incomplete clinical data.

Introduction

Predicting the time until a specific clinical event—such as disease recurrence or mortality—is fundamental to tailoring patient treatment. Recent advancements have seen the rise of multimodal deep learning models that fuse high-resolution pathology images with high-dimensional genomic data. While these models offer superior predictive power, they typically assume the availability of paired data for every patient.

In real-world clinical pipelines, data incompleteness is a pervasive issue; a patient may have a biopsy slide but lack genomic sequencing, or vice versa. Existing approaches to address this, such as knowledge distillation or direct feature reconstruction, often struggle with the significant heterogeneity between visual and molecular data. ProSurv addresses these challenges by treating survival prediction as an adaptive task, leveraging global statistics to bridge the gap between disparate modalities.

Methodology

The ProSurv framework is built upon three core pillars designed to facilitate robust cross-modal knowledge transfer.

1. Intra-Modal Prototype Update

To capture the underlying distribution of features relative to patient outcomes, ProSurv maintains two modality-specific "prototype banks" (one for pathology and one for genomics). These banks store representative feature vectors, or prototypes, corresponding to discrete survival time intervals (bins). The model employs risk contrastive learning to refine these prototypes, ensuring that features from the same survival bin are clustered together while those from different bins are pushed apart. This creates a structured "memory" of high-risk versus low-risk signatures for each modality.

2. Prototype-Guided Cross-Modal Translation

The central innovation of ProSurv is its ability to "fill in the blanks" when data is missing. If only pathology data is available, the model uses the extracted image features as a "query" in a cross-attention mechanism. It probes the genomic prototype bank to identify relevant molecular patterns associated with the observed pathology. This process generates a synthetic genomic feature that approximates the missing information, effectively translating visual evidence into molecular knowledge.

3. Knowledge-Enhanced Learning

During the training phase, the model utilizes an alignment loss ($L_{align}$) to ensure that translated features closely resemble their real-world counterparts. At inference time, the framework is truly adaptive: it can fuse available original features with translated features to make a final prediction, regardless of whether the input is pathology-only, gene-only, or multimodal.

Experimental Results

Datasets and Metrics

The researchers evaluated ProSurv on four public datasets from The Cancer Genome Atlas (TCGA): Breast Invasive Carcinoma (BRCA), Bladder Urothelial Carcinoma (BLCA), Stomach Adenocarcinoma (STAD), and Colon and Rectum Adenocarcinoma (CRAD). Performance was measured using the C-index (Concordance Index), which quantifies the model's ability to correctly rank patients by their survival times.

Performance Analysis

ProSurv was compared against 11 state-of-the-art methods, including MCAT, SurvPath, and G-HANet. Key findings include:

  • Multimodal Superiority: ProSurv achieved an overall C-index of 0.645, outperforming all baseline multimodal methods.
  • Unimodal Enhancement: In pathology-only scenarios, ProSurv reached a C-index of 0.640, significantly higher than G-HANet (0.617) and MCAT (0.614). This indicates that the translated genomic knowledge provides a meaningful performance boost even when the actual genomic data is absent.
  • Robustness: The model maintained stable performance even when the training data was 50% unimodal, demonstrating its resilience to high rates of missing data.

Discussion and Insights

ProSurv’s success stems from its use of prototypes as global statistics. Rather than attempting a direct, high-variance mapping between a specific image patch and a gene sequence, the model maps features to a bank of learned global statistics. This makes the translation process more robust to the inherent noise in histo-genomic data.

Furthermore, the model incorporates event-aware sampling. By recognizing that "censored" data (patients who left the study before an event) still provides a lower bound for survival, the sampling strategy treats prototypes following the censoring time as positive samples. This ensures the model utilizes the maximum amount of information available in clinical datasets.

Limitations and Future Work

While ProSurv represents a significant step forward, several areas for improvement remain. The discretization of survival time into bins, while computationally efficient, may limit the precision of the model compared to continuous-time approaches. Additionally, while the model's performance is high, further research is needed to interpret the biological significance of the translated features—specifically, identifying which visual morphologies trigger specific synthetic genomic signatures. Future iterations may also explore fine-tuning foundation models, such as UNI, specifically for the prototype-guided translation task to further enhance feature extraction.

Summary updated

BibTeX
@inproceedings{liu2025prototype,
  author = {Fengchun Liu and Linghan Cai and Zhikang Wang and Zhiyuan Fan and Jin-gang Yu and Hao Chen and Yongbing Zhang},
  title = {{Prototype-Guided Cross-Modal Knowledge Enhancement for Adaptive Survival Prediction}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2025},
  doi = {10.1007/978-3-032-04978-0\_50},
}
Figure