Conditional Segmentation in Lieu of Image Registration

Yipeng Hu, Eli Gibson, Dean C. Barratt, Mark Emberton, J. Alison Noble, Tom Vercauteren

TL;DR

This paper introduces a novel "conditional segmentation" approach that directly predicts the location of specific regions of interest (ROIs) in a target image, bypassing the need for traditional image registration. This method significantly improves accuracy for propagating ROIs, particularly in medical imaging applications like prostate cancer treatment planning, by avoiding the over-constraining nature of dense deformation prediction.

What Problem Does This Paper Address?

Medical image registration is crucial for aligning images from different modalities or time points, enabling tasks like surgical guidance. Traditional methods, and even recent deep learning approaches, focus on predicting a dense spatial transformation (like a displacement field) that maps every point from a "moving" image to a "fixed" image. However, in many clinical scenarios, the ultimate goal isn't to know the exact transformation of every voxel, but rather to accurately locate specific, often sparse, regions of interest (ROIs) – like tumors or critical anatomical structures – in the target image. The paper argues that the dense, spatially regularized transformations predicted by current methods can be overly restrictive and may not be optimal for precisely localizing these ROIs, especially when these ROIs have complex or non-linear relationships between images. This can lead to less accurate ROI propagation, which is critical for interventions.

Methods

The core innovation is a conditional segmentation paradigm. Instead of predicting a full spatial transformation, the proposed network directly learns to segment the target ROI in the fixed image, conditioned on the input moving image and the corresponding ROI in the moving image. This essentially reframes the problem from "where does this point go?" to "is this point part of the ROI in the target image, given the source ROI?".

Key aspects of the method:

  • Input: The network takes the moving image, the fixed image, and the moving ROI as input.
  • Output: It outputs a probability map for each voxel in the fixed image, indicating whether it belongs to the propagated ROI.
  • Architecture: It's built upon standard image segmentation networks (like a 3D U-Net), with minimal adaptation to concatenate the image pair and the moving ROI as input.
  • Training: A supervised approach is used, leveraging ground-truth ROIs in the fixed images. A weighted cross-entropy loss function is employed, with weighting to account for foreground/background voxel imbalance.
  • No Deformation Regularization: Crucially, this method does not require deformation regularization terms (like L2-norm of DDF or bending energy) that are common in registration networks. This is a key differentiator, as it allows the network to learn more complex, non-linear mappings without being constrained by assumptions of smoothness.

Datasets

The study uses a dataset of 115 pairs of T2-weighted MR and 3D transrectal ultrasound (TRUS) images from 80 prostate cancer patients. These images were acquired during TRUS-guided biopsy or therapy procedures.

  • ROIs: A total of 910 pairs of corresponding anatomical ROIs were manually labeled and verified by expert observers. These include landmarks like the prostate apex/base, urethra, visible lesions, gland zonal separations, vas deferens, seminal vesicles, and other patient-specific features like calcifications and cysts.
  • Benchmark: The dataset is used to compare the proposed method against a previously proposed weakly-supervised registration network (Hu et al., 2018) that predicts a dense displacement field (DDF) and then warps ROIs. The authors emphasize that the same types of image and ROI data were used for training both networks to ensure a fair comparison.

Key Insights and Ideas

  • Reframing Registration as Segmentation: The most significant insight is that for ROI propagation tasks, directly learning the segmentation of the target ROI is more effective than learning a full spatial transformation. This shifts the focus from global alignment to localizing specific features.
  • Avoiding Deformation Regularization Benefits: By omitting deformation regularization, the conditional segmentation approach can learn more complex, non-linear, and even non-smooth mappings. This is particularly beneficial for anatomical structures that exhibit high variability or undergo significant changes during procedures.
  • Reduced Bias: The bias-variance analysis suggests that the conditional segmentation method exhibits significantly lower bias compared to DDF-predicting registration networks. This lower bias is a primary contributor to the improved accuracy.
  • Potential for Better Generalization: The paper hypothesizes that avoiding deformation regularization might lead to better generalizability to out-of-sample ROIs, as the network is less constrained by assumptions about the deformation field.

Key Results

  • Improved Target Registration Error (TRE): The proposed conditional segmentation method achieved a median TRE of 2.1 mm (25th-75th percentile: 1.4-3.5 mm). This is a significant improvement over the DDF-predicting registration network, which achieved a median TRE of 3.2 mm (2.3-6.4 mm). This represents a >34% reduction in TRE.
  • Ad Hoc Landmark Performance: Even for manually selected "ad hoc" landmarks not explicitly represented in the training data, the conditional segmentation method showed a lower median TRE (2.8 mm vs. 4.2 mm for the registration network).
  • Bias Reduction: The median bias (d_bias) was considerably lower for the conditional segmentation network across all training set sizes, suggesting it's a more accurate predictor of ROI centroids.
  • No Significant Variance Difference: While bias was clearly reduced, detecting significant differences in variance between the two networks was difficult due to small sample sizes in the cross-validation experiments.

Limitations and Future Work

  • Training Data Size: The paper notes that detecting differences in variance between the two networks was difficult due to the small sample sizes used in the repeated cross-validation. Future work could explore larger sample sizes to better understand variance differences.
  • Bias-Variance Trade-off: While the conditional segmentation shows lower bias, the paper doesn't definitively conclude on the overall bias-variance trade-off without more robust variance estimation.
  • Generalizability to Other Modalities/Applications: The study focuses on MR-to-ultrasound prostate imaging. Further validation on other medical imaging modalities and anatomical regions would be valuable.
  • Clinical Utility: While TRE is a good indicator, further studies could directly assess the clinical impact of this improved ROI propagation on downstream tasks like surgical planning and execution.

Summary updated

Abstract

Classical pairwise image registration methods search for a spatial transformation that optimises a numerical measure that indicates how well a pair of moving and fixed images are aligned. Current learning-based registration methods have adopted the same paradigm and typically predict, for any new input image pair, dense correspondences in the form of a dense displacement field or parameters of a spatial transformation model. However, in many applications of registration, the spatial transformation itself is only required to propagate points or regions of interest (ROIs). In such cases, detailed pixel- or voxel-level correspondence within or outside of these ROIs often have little clinical value. In this paper, we propose an alternative paradigm in which the location of corresponding image-specific ROIs, defined in one image, within another image is learnt. This results in replacing image registration by a conditional segmentation algorithm, which can build on typical image segmentation networks and their widely-adopted training strategies. Using the registration of 3D MRI and ultrasound images of the prostate as an example to demonstrate this new approach, we report a median target registration error (TRE) of 2.1 mm between the ground-truth ROIs defined on intraoperative ultrasound images and those propagated from the preoperative MR images. Significantly lower (>34%) TREs were obtained using the proposed conditional segmentation compared with those obtained from a previously-proposed spatial-transformation-predicting registration network trained with the same multiple ROI labels for individual image pairs. We conclude this work by using a quantitative bias-variance analysis to provide one explanation of the observed improvement in registration accuracy.
BibTeX
@inproceedings{hu2019conditional,
  author = {Yipeng Hu and Eli Gibson and Dean C. Barratt and Mark Emberton and J. Alison Noble and Tom Vercauteren},
  title = {{Conditional Segmentation in Lieu of Image Registration}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2019},
  url = {https://openreview.net/forum?id=Nhc1KD5jSh},
}
Figure