CorrSigNet: Learning CORRelated Prostate Cancer SIGnatures from Radiology and Pathology Images for Improved Computer Aided Diagnosis

Wei Shao, Indrani Bhattacharya, Arun Seetharaman, Wei Shao 0008, Rewa Sood, Christian A. Kunder, Richard E. Fan, Simon John Christoph Soerensen, Jeffrey B. Wang, Pejman Ghanouni, Nikola C. Teslovich, James D. Brooks, Geoffrey A. Sonn, Mirabela Rusu

TL;DR

This paper introduces CorrSigNet, a novel deep learning framework that bridges the gap between radiology (MRI) and pathology images to improve prostate cancer detection. By learning correlated "signatures" between these modalities, CorrSigNet achieves state-of-the-art accuracy in localizing cancerous regions, even for subtle or MRI-invisible tumors.

What Problem Does This Paper Address?

Prostate cancer diagnosis and staging heavily rely on multi-parametric Magnetic Resonance Imaging (mp-MRI). However, interpreting these images can be challenging due to subtle visual cues, leading to significant variability among radiologists and missed diagnoses. While machine learning models have been developed to aid in this process, they often rely solely on MRI features, neglecting the crucial biological information present in histopathology. This disconnect means current models can miss cancers that are not clearly visible on MRI, and they don't fully capture the disease's true extent as revealed by tissue analysis. Furthermore, existing methods for correlating MRI with pathology often use less precise techniques like cognitive registration, which can introduce inaccuracies.

Methods

CorrSigNet employs a two-step approach:

  1. Common Representation Learning: This is the core innovation. The model uses spatially aligned MRI and histopathology images from the same patients to learn "correlated signatures." It leverages a modified Common Representation Learning (CorrNet) architecture. This network takes features extracted from both MRI (T2W and ADC sequences) and histopathology images and learns a shared, lower-dimensional representation. The key idea is that these learned representations capture aspects of the tumor biology visible in pathology that might be subtle or absent in MRI alone. This step is crucial because it allows the model to learn what MRI features should look like if they correspond to cancerous tissue as seen under a microscope.
  2. Predictive Model Training: Once the correlated MRI signatures are learned, the model uses these signatures to train a Convolutional Neural Network (CNN) for cancer localization. This second step is critical because it allows the model to predict cancer on new MRI scans without needing the corresponding histopathology images (which are only available after surgery). The paper explores two modified Holistically Nested Edge Detection (HED) architectures: HED-3 (using only correlated MRI features) and HED-branch-3 (using correlated MRI features combined with original T2W and ADC images).

Datasets

The study utilized a dataset of 95 prostatectomy patients. For each patient, pre-operative mp-MRI (T2W and ADC) and post-operative digitized histopathology images were collected. A key aspect of their dataset preparation was the use of custom 3D printed molds to ensure that the excised prostate tissue was sectioned in the same plane as the T2W MRI. This meticulous alignment, facilitated by the RAPSODI registration platform, allowed for pixel-level correlation analysis. The dataset was split into training (66 patients), validation (9 patients), and testing (20 patients) sets. The test set comprised 139 slices, containing 24 cancerous lesions and approximately 1.12 million pixels.

Key Insights and Ideas

  • Bridging the Modality Gap: The most significant insight is the successful learning of "correlated signatures" that link MRI appearance to underlying pathology. This allows the model to infer biological characteristics from MRI that might otherwise be missed.
  • Pathology as a "Ground Truth" for MRI Features: By using histopathology to guide the learning of MRI features, the model effectively learns what subtle MRI patterns mean in terms of cancer biology, rather than just learning patterns that are statistically associated with labels derived solely from MRI.
  • MRI-Invisible Cancer Detection: The correlated features are shown to be particularly effective at identifying subtle cancerous regions that are difficult to discern on MRI alone, or even "MRI-invisible" cancers.
  • Generalizability: The learned correlated features can be extracted from MRI scans of new patients without requiring histopathology, making the approach applicable for pre-surgical diagnosis.

Key Results

  • Pixel-Level Performance: CorrSigNet models consistently outperformed the current state-of-the-art method [4]. The best performing model, CorrSigNet(T2W, ADC, 5), achieved a per-pixel AUC of 0.86, compared to 0.80 for the baseline.
  • Lesion-Level Performance: The CorrSigNet(T2W, ADC, 5) model achieved a per-lesion AUC of 0.96 ± 0.07, surpassing the baseline's 0.92 ± 0.09.
  • Qualitative Improvements: Visual examples demonstrate that CorrSigNet can identify cancerous regions missed by the baseline method, shows better overlap with ground truth cancer labels, and exhibits fewer false positives.

Limitations and Future Work

  • Dataset Size: While 95 patients is a reasonable starting point, the authors acknowledge that augmented datasets and a cross-validation framework could further boost performance.
  • Focus on Intra-Prostate Cancer: The current analysis was focused only on cancers within the prostate, excluding extra-prostatic extensions.
  • Future Work: The authors plan to conduct experiments with augmented datasets and explore a cross-validation framework to enhance model robustness and performance. They also aim to further investigate the capability of correlated features in capturing tumor biology information.

Summary updated

Abstract

Magnetic Resonance Imaging (MRI) is widely used for screening and staging\nprostate cancer. However, many prostate cancers have subtle features which are\nnot easily identifiable on MRI, resulting in missed diagnoses and alarming\nvariability in radiologist interpretation. Machine learning models have been\ndeveloped in an effort to improve cancer identification, but current models\nlocalize cancer using MRI-derived features, while failing to consider the\ndisease pathology characteristics observed on resected tissue. In this paper,\nwe propose CorrSigNet, an automated two-step model that localizes prostate\ncancer on MRI by capturing the pathology features of cancer. First, the model\nlearns MRI signatures of cancer that are correlated with corresponding\nhistopathology features using Common Representation Learning. Second, the model\nuses the learned correlated MRI features to train a Convolutional Neural\nNetwork to localize prostate cancer. The histopathology images are used only in\nthe first step to learn the correlated features. Once learned, these correlated\nfeatures can be extracted from MRI of new patients (without histopathology or\nsurgery) to localize cancer. We trained and validated our framework on a unique\ndataset of 75 patients with 806 slices who underwent MRI followed by\nprostatectomy surgery. We tested our method on an independent test set of 20\nprostatectomy patients (139 slices, 24 cancerous lesions, 1.12M pixels) and\nachieved a per-pixel sensitivity of 0.81, specificity of 0.71, AUC of 0.86 and\na per-lesion AUC of $0.96 \\pm 0.07$, outperforming the current state-of-the-art\naccuracy in predicting prostate cancer using MRI.\n
BibTeX
@inproceedings{shao2020corrsignet,
  author = {Wei Shao and Indrani Bhattacharya and Arun Seetharaman and Wei Shao and Rewa Sood and Christian A. Kunder and Richard E. Fan and Simon John Christoph Soerensen and Jeffrey B. Wang and Pejman Ghanouni and Nikola C. Teslovich and James D. Brooks and Geoffrey A. Sonn and Mirabela Rusu},
  title = {{CorrSigNet: Learning CORRelated Prostate Cancer SIGnatures from Radiology and Pathology Images for Improved Computer Aided Diagnosis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59713-9\_31},
}
Figure