Learning Visual Features by Colorization for Slide-Consistent Survival Prediction from Whole Slide Images

Lei Fan, Arcot Sowmya, Erik Meijering, Yang SongMICCAI 2021 · OpenReview

Abstract

Recent deep learning techniques have shown promising performance on survival prediction from Whole Slide Images (WSIs). These methods are often based on multiple-step frameworks including patch sampling, feature extraction, and feature aggregation. However, feature extraction typically relies on handcrafted features or Convolutional Neural Networks (CNNs) pretrained on ImageNet without fine-tuning, thus leading to suboptimal performance. Besides, to aggregate features, previous studies focus on WSI-level survival prediction but ignore the heterogeneous information that is present in multiple WSIs acquired for the same patient. To address the above challenges, we propose a survival prediction model that exploits heterogeneous features at the patient-level. Specifically, we introduce colorization as the pretext task to train the CNNs which are tailored for extracting features from patches of WSIs. In addition, we develop a patient-level framework integrating multiple WSIs for survival prediction with consistency and ranking losses. Extensive experiments show that our model achieves state-of-the-art performance on two large-scale public datasets.
BibTeX
@inproceedings{fan2021learning,
  author = {Lei Fan and Arcot Sowmya and Erik Meijering and Yang Song},
  title = {{Learning Visual Features by Colorization for Slide-Consistent Survival Prediction from Whole Slide Images}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  url = {https://openreview.net/forum?id=qDs_PzbGUBD},
}
Figure