Image registration of in vivo micro-ultrasound and ex vivo pseudo-whole mount histopathology images of the prostate: A proof-of-concept study

Muhammad Imran, Brianna Nguyen, Jake Pensa, Sara M. Falzarano, Anthony E. Sisk, Muxuan Liang, John Michael DiBianco, Li-Ming Su, Yuyin Zhou, Jason P. Joseph, Wayne G. Brisbane, Wei ShaoBiomedical Signal Processing and Control 2023 · DOI · OpenReview

TL;DR

This paper introduces a novel semi-automated pipeline for precisely aligning in-vivo micro-ultrasound (micro-US) images with ex-vivo whole-mount prostate histopathology. This breakthrough enables the mapping of precise cancer annotations from detailed pathology slides onto real-time ultrasound images, paving the way for improved machine learning diagnostics and urologist training.

What Problem Does This Paper Address?

Accurate diagnosis and treatment of prostate cancer, especially for smaller tumors, heavily rely on imaging guidance. While MRI-ultrasound fusion biopsy is a standard, it's hampered by the high cost of MRI, the need for specialized radiologists, and potential fusion errors. Micro-ultrasound offers a cost-effective, high-resolution alternative, but interpreting its subtle grayscale changes for cancer detection is challenging for clinicians. To overcome this, researchers need a way to correlate the "ground truth" cancer locations from detailed histopathology slides with micro-US images. However, directly registering these modalities is difficult due to differences in acquisition planes (oblique for micro-US, axial for histopathology) and significant tissue deformation in histopathology samples. Existing registration methods for MRI-histopathology are not directly applicable to micro-US.

Methods

The authors developed a semi-automated pipeline with several key steps:

  1. Pseudo-Whole Mount Histopathology Reconstruction: Histopathology slides, often fragmented due to the size of prostate specimens, are stitched together using the AutoSticher tool to create a complete "pseudo-whole mount" image.
  2. 3D Micro-US Volume Reconstruction: Serial micro-US images acquired in oblique planes are transformed into a 3D volume oriented in the axial plane, matching the histopathology orientation. This involves estimating voxel positions based on probe rotation angles and image dimensions.
  3. Slice Correspondence Estimation: A crucial manual step involves identifying corresponding slices between the pseudo-whole mount histopathology images and the reconstructed axial micro-US volume. This is achieved by visually matching prominent anatomical features.
  4. Two-Stage Image Registration: For each corresponding pair of histopathology and micro-US slices, a two-stage registration process is employed:
  5. Affine Registration: An initial rigid and scaling transformation is applied, guided by prostate segmentations and using sum of squared differences as the loss function.
  6. Deformable Registration: A more flexible, free-form deformation is then applied to account for tissue warping, using masked prostate images and Mattes mutual information as the loss function. A multi-resolution pyramid approach is used for efficiency.

Datasets

The study utilized a dataset of 18 patients who underwent radical prostatectomy at the University of Florida. For each patient, both in-vivo micro-US images and ex-vivo pseudo-whole mount histopathology images were acquired. The histopathology images were annotated by an expert pathologist for cancer outlines. The authors also mention that they will make their aligned micro-US and histopathology images publicly available, creating a valuable resource for future research.

Key Insights and Ideas

  • Bridging the Modality Gap: The core innovation is the successful registration between micro-US and histopathology, two modalities with fundamentally different acquisition characteristics and orientations. This is a significant step towards creating a comprehensive dataset for micro-US analysis.
  • Proof-of-Concept Success: The study demonstrates that despite the inherent challenges of tissue deformation and differing acquisition planes, accurate registration is achievable, validating the pipeline's potential.
  • Enabling Machine Learning: By mapping precise cancer outlines from histopathology onto micro-US images, this work directly facilitates the development and training of machine learning algorithms for automated cancer detection and grading on micro-US.
  • Improving Clinical Interpretation: The aligned images can serve as a valuable training tool for urologists, helping them better interpret the subtle signs of cancer in high-resolution micro-US scans.

Key Results

The registration pipeline achieved promising quantitative results:

  • Dice Coefficient: A mean Dice coefficient of 0.965 was reported for prostate boundary alignment, indicating excellent overlap between segmented prostate regions.
  • Hausdorff Distance: The mean Hausdorff distance was 1.77 mm, demonstrating high accuracy in aligning prostate boundaries.
  • Urethra Deviation: A mean urethra deviation of 2.20 mm was achieved, comparable to state-of-the-art MRI-histopathology registration.
  • Landmark Error: The mean landmark error was 2.71 mm, also comparable to existing MRI-histopathology methods, validating the alignment of anatomical features.

Qualitatively, the results showed good alignment of prostate boundaries, urethra, and cancer regions between the deformed histopathology images and the fixed micro-US images.

Limitations and Future Work

The authors acknowledge several limitations:

  • Small Dataset: The study was a proof-of-concept with a relatively small dataset of 18 patients.
  • Manual Steps: The pipeline still requires several manual interventions, including pseudo-whole mount histopathology reconstruction, prostate segmentation, and estimation of slice correspondences.

Future work will focus on:

  • Expanding the Dataset: Collecting data from a larger cohort of patients to further validate the pipeline.
  • Automating Manual Steps: Developing deep learning techniques to automate the manual steps, leading to a more efficient and robust registration process.
  • Wider Adoption: Contributing to the advancement and wider adoption of micro-US as a diagnostic tool.

Summary updated

Abstract

Highlights•First method for registering micro-ultrasound and whole-mount histopathology images.•Cross-validation on 18 patients demonstrates the accuracy of our registration method.•Cancer labels from image registration enable cancer detection on micro-ultrasound.•Our dataset and code are publicly available.
BibTeX
@article{imran2023image,
  author = {Muhammad Imran and Brianna Nguyen and Jake Pensa and Sara M. Falzarano and Anthony E. Sisk and Muxuan Liang and John Michael DiBianco and Li-Ming Su and Yuyin Zhou and Jason P. Joseph and Wayne G. Brisbane and Wei Shao},
  title = {{Image registration of in vivo micro-ultrasound and ex vivo pseudo-whole mount histopathology images of the prostate: A proof-of-concept study}},
  journal = {Biomedical Signal Processing and Control},
  year = {2023},
  doi = {10.1016/j.bspc.2024.106657},
}
Figure