Weakly Supervised Registration of Prostate MRI and Histopathology Images

Wei Shao, Wei Shao 0008, Indrani Bhattacharya, Simon John Christoph Soerensen, Christian A. Kunder, Jeffrey B. Wang, Richard E. Fan, Pejman Ghanouni, James D. Brooks, Geoffrey A. Sonn, Mirabela RusuMICCAI 2021 · DOI

TL;DR

This paper introduces a novel weakly supervised deep learning approach for aligning prostate MRI and histopathology images. By eliminating the need for manual prostate segmentations during inference, this method significantly simplifies and improves the accuracy of mapping cancer information from pathology slides onto MRI scans, aiding in better diagnosis and treatment planning.

What Problem Does This Paper Address?

Prostate cancer diagnosis and management are hampered by low agreement among radiologists interpreting MRI scans. This is partly due to the difficulty in precisely correlating the extent of cancer seen on surgical histopathology slides with the corresponding MRI. Image registration is a key technique to bridge this gap by mapping the "ground truth" from pathology onto MRI. However, existing automated registration methods often require manual segmentation of the prostate in both modalities, which is a time-consuming, error-prone, and not always feasible step, especially since pathologists don't typically segment prostates. This paper addresses the critical need for an automated registration method that bypasses the requirement for manual prostate segmentations during the actual registration process.

Methods

The core of this work is a weakly supervised deep learning framework for both affine and deformable registration. The authors leverage a modified VGG16 network for feature extraction from both the fixed MRI and moving histopathology images. Crucially, the network is trained using a combination of:

  • Manual prostate segmentations: Used during training to guide the network to align prostate boundaries and local features.
  • Mono-modal synthetic image pairs: These are generated by applying random transformations to real images, allowing the network to learn to correct for these transformations without relying on multi-modal appearance differences.
  • Unmasked real multimodal image pairs: Used in conjunction with manual masks during training.

The loss function is a combination of:

  • Intensity Loss ($L_{int}$): An unsupervised loss that encourages alignment of local features by comparing transformed synthetic images with their original counterparts.
  • Segmentation Loss ($L_{seg}$): A Dice coefficient loss that directly penalizes misalignment of prostate boundaries.
  • Regularization Loss ($L_{reg}$): A loss that ensures the smoothness of the nonrigid transformations, preventing unrealistic deformations.

The affine and deformable registration networks are trained simultaneously, with the affine transformation output serving as an initialization for the deformable one. A key innovation is that while segmentations are used for training, they are not required for inference (i.e., registering new, unseen images).

Datasets

The study utilizes two cohorts:

  • Internal Cohort: 161 patients with pre-operative T2-weighted MRI and whole-mount digital histopathology images from radical prostatectomy.
  • External Validation Cohort: 22 patients from the publicly available TCIA "Prostate-MRI" dataset.

For both cohorts, manual segmentations of the prostate, urethra, and anatomical landmarks were performed on MRI and histopathology images. Cancerous regions on histopathology were also manually segmented.

Key Insights and Ideas

  • Weak Supervision is Key: The most significant insight is demonstrating that effective registration can be achieved without manual segmentations at inference time. This dramatically lowers the barrier to entry for using such registration techniques.
  • Synthetic Data for Robustness: Using synthetic mono-modal image pairs for training helps the network learn the underlying geometric transformations independent of the inherent differences in appearance between MRI and histopathology.
  • Combined Loss for Comprehensive Alignment: The multi-component loss function (intensity, segmentation, and regularization) effectively addresses different aspects of the registration problem, leading to more accurate and robust results.
  • Generalization Across Datasets: The model trained on an internal cohort shows good performance on an external, publicly available dataset, suggesting its generalizability.

Key Results

The proposed weakly supervised method significantly outperforms the state-of-the-art ProsRegNet method, particularly when segmentations are not used during testing.

  • Quantitative Improvements:
    • Dice Coefficient: Achieved an average of 0.90 (±0.02) for the internal cohort and 0.89 (±0.03) for the external cohort, compared to ProsRegNet's 0.80 (±0.06) and 0.82 (±0.04) respectively. This indicates better overlap of prostate boundaries.
  • Hausdorff Distance: Showed significantly lower distances (4.38 ±0.64 internal, 5.01 ±0.97 external) compared to ProsRegNet (7.76 ±2.00 internal, 8.00 ±2.12 external), meaning more accurate boundary alignment.
  • Urethra Deviation and Landmark Error: The proposed method also demonstrated superior alignment of local prostate features.
  • Qualitative Results: Visual comparisons show that the proposed method accurately aligns prostate boundaries and local features (like the urethra and potential cancerous regions), whereas ProsRegNet showed only marginal improvement over the initial misalignment.

Limitations and Future Work

  • Spectrum Bias: The authors acknowledge that their internal cohort consists of patients who underwent radical prostatectomy. This might introduce a spectrum bias, as these patients likely have more aggressive disease compared to those considered for active surveillance. Future work could explore generalization to these other populations.
  • Grade as Surrogate: For aggressive disease, the study used pathological grade as a surrogate, which is a reasonable approach but might not capture all nuances of cancer aggressiveness.
  • Future Directions: The paper highlights that accurate cancer labels on MRI, facilitated by their registration method, can significantly aid clinical care and the development of deep learning approaches for early detection of aggressive prostate cancers on preoperative MRI. This suggests a clear path for future applications and research.

Summary updated

BibTeX
@inproceedings{shao2021weakly,
  author = {Wei Shao and Wei Shao and Indrani Bhattacharya and Simon John Christoph Soerensen and Christian A. Kunder and Jeffrey B. Wang and Richard E. Fan and Pejman Ghanouni and James D. Brooks and Geoffrey A. Sonn and Mirabela Rusu},
  title = {{Weakly Supervised Registration of Prostate MRI and Histopathology Images}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87202-1\_10},
}
Figure