CorrSigNet: Learning CORRelated Prostate Cancer SIGnatures from\n Radiology and Pathology Images for Improved Computer Aided Diagnosis

Indrani Bhattacharya, Arun Seetharaman, Wei Shao, Rewa Sood, Christian A. Kunder, Richard E. Fan, Simon John Christoph Soerensen, Jeffrey B. Wang, Pejman Ghanouni, Nikola C. Teslovich, James D. Brooks, Geoffrey A. Sonn, Mirabela Rusu

TL;DR

CorrSigNet is a novel deep learning framework that improves prostate cancer detection on MRI by "teaching" the model to recognize MRI signatures that correlate with gold-standard histopathology. By leveraging surgical data during training to find a common representation between radiology and pathology, the model achieves state-of-the-art accuracy in localizing lesions—even when only MRI is available for a new patient.

What Problem Does This Paper Address?

Prostate cancer is a leading cause of cancer-related deaths, and while Multi-Parametric MRI (mpMRI) is the standard for screening, it isn't perfect. Radiologists often struggle with "MRI-invisible" or subtle lesions, leading to low inter-reader agreement (0.46–0.78) and high variability in diagnosis.

Existing machine learning models typically learn only from MRI features. The authors argue these models are "agnostic to the biology" of the tumor. Because MRI often underestimates tumor size and misses subtle cellular changes, models trained solely on MRI labels (which are often retrospectively outlined by radiologists) inherit these limitations. The challenge is: how do we give an MRI-based model the "ground truth" insights of a pathologist without requiring every patient to undergo surgery first?

Methods

The authors propose a two-step approach called CorrSigNet:

  1. Common Representation Learning (CRL): The team used a Correlational Neural Network (CorrNet). During the training phase, the model is fed both MRI features (T2-weighted and ADC maps) and digitized histopathology images from patients who had their prostates removed (prostatectomy). The CorrNet learns a hidden "common space" where MRI features and pathology features are highly correlated.
  2. Cancer Localization: Once the model learns how to project MRI data into this pathology-informed space, the histopathology data is discarded. These "correlated MRI features" are then used to train a modified Holistically Nested Edge Detection (HED) network.

The HED architecture was modified into two versions: HED-3 (using only correlated features) and HED-branch-3 (combining correlated features with raw MRI intensities). These models process three adjacent slices simultaneously to maintain 3D volumetric continuity, which helps reduce false positives.

Datasets

The study utilized a unique, high-quality dataset of 95 prostate surgery patients:

  • Imaging: Pre-operative T2-weighted MRI and Apparent Diffusion Coefficient (ADC) maps.
  • Ground Truth: Post-operative digitized histopathology images.
  • The "Secret Sauce": To ensure the MRI and pathology images matched perfectly, the researchers used custom 3D-printed molds of each patient's prostate. This allowed the surgical tissue to be sliced in the exact same plane as the MRI, enabling pixel-level registration (using the RAPSODI platform).
  • Split: 66 patients for training, 9 for validation, and 20 for an independent test set.

Key Insights and Ideas

  • Pathology-Informed MRI: The core "aha moment" is that pathology shouldn't just be a label (cancer vs. no cancer); it should be a teacher. By forcing the model to find MRI features that correlate with cellular-level pathology, the model learns to "see" biological signatures that are otherwise invisible to the human eye on a standard scan.
  • Feature Projection: At inference time (for a new patient), the model doesn't need pathology. It simply projects the new MRI data into the learned "correlated space," effectively looking for the MRI signatures it knows are associated with cancer cells.
  • 3D Context Matters: By using a three-slice input (the target slice plus its neighbors), the model significantly reduced "speckle" noise and false positives, ensuring that predicted lesions have realistic 3D shapes.

Key Results

CorrSigNet outperformed the current state-of-the-art (SOTA) methods across multiple metrics:

  • Per-Pixel Performance: Achieved an AUC of 0.86, a sensitivity of 0.81, and a specificity of 0.71.
  • Per-Lesion Performance: Achieved a remarkably high AUC of 0.96 ± 0.07.
  • Qualitative Success: The model successfully identified smaller, subtle cancerous regions that the previous SOTA model missed and showed significantly better overlap with the actual ground-truth labels provided by pathologists.

Limitations and Future Work

  • Dataset Size: While the pixel-level training provides over a million samples, the patient count (95) is relatively small. The authors plan to test the model on larger, more diverse datasets.
  • Scope: The current study focused only on cancers located inside the prostate gland, ignoring extra-prostatic extensions (cancer that has spread just outside the boundary).
  • Validation: Future work will involve cross-validation frameworks and potentially testing how this model performs across different MRI scanner manufacturers, as MRI intensities can vary between machines.

Summary updated

Abstract

Magnetic Resonance Imaging (MRI) is widely used for screening and staging\nprostate cancer. However, many prostate cancers have subtle features which are\nnot easily identifiable on MRI, resulting in missed diagnoses and alarming\nvariability in radiologist interpretation. Machine learning models have been\ndeveloped in an effort to improve cancer identification, but current models\nlocalize cancer using MRI-derived features, while failing to consider the\ndisease pathology characteristics observed on resected tissue. In this paper,\nwe propose CorrSigNet, an automated two-step model that localizes prostate\ncancer on MRI by capturing the pathology features of cancer. First, the model\nlearns MRI signatures of cancer that are correlated with corresponding\nhistopathology features using Common Representation Learning. Second, the model\nuses the learned correlated MRI features to train a Convolutional Neural\nNetwork to localize prostate cancer. The histopathology images are used only in\nthe first step to learn the correlated features. Once learned, these correlated\nfeatures can be extracted from MRI of new patients (without histopathology or\nsurgery) to localize cancer. We trained and validated our framework on a unique\ndataset of 75 patients with 806 slices who underwent MRI followed by\nprostatectomy surgery. We tested our method on an independent test set of 20\nprostatectomy patients (139 slices, 24 cancerous lesions, 1.12M pixels) and\nachieved a per-pixel sensitivity of 0.81, specificity of 0.71, AUC of 0.86 and\na per-lesion AUC of $0.96 \\pm 0.07$, outperforming the current state-of-the-art\naccuracy in predicting prostate cancer using MRI.\n
BibTeX
@inproceedings{bhattacharya2020corrsignet,
  author = {Indrani Bhattacharya and Arun Seetharaman and Wei Shao and Rewa Sood and Christian A. Kunder and Richard E. Fan and Simon John Christoph Soerensen and Jeffrey B. Wang and Pejman Ghanouni and Nikola C. Teslovich and James D. Brooks and Geoffrey A. Sonn and Mirabela Rusu},
  title = {{CorrSigNet: Learning CORRelated Prostate Cancer SIGnatures from\textbackslash\{\}n Radiology and Pathology Images for Improved Computer Aided Diagnosis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59713-9\_31},
}
Figure