Are Registration Uncertainty and Error Monotonically Associated?

Jie Luo, Sarah F. Frisken, Duo Wang, Alexandra J. Golby, Masashi Sugiyama, William M. Wells Iii

TL;DR

This paper investigates whether the uncertainty provided by Gaussian Process (GP) registration in neurosurgery can reliably predict registration error. They found a weak-to-moderate positive correlation, suggesting that while uncertainty might offer some guidance, it's not a perfect proxy for error in clinical practice. This has implications for the trustworthiness of current image-guided surgery systems.

What Problem Does This Paper Address?

Image-guided neurosurgery (IGN) relies on accurately aligning pre-operative images (like MRI) with the patient's anatomy during surgery. While rigid registration is common, non-rigid registration is more desirable for accounting for tissue deformation, especially after tumor resection. However, predicting and validating non-rigid registration error is challenging. Surgeons often face discrepancies between aligned images and struggle to differentiate between actual tissue movement and registration mistakes. This uncertainty can lead to critical errors. Probabilistic Image Registration (PIR) methods offer "registration uncertainty" as a potential surrogate for error. The intuitive assumption is that high uncertainty means high error, but this monotonic relationship hasn't been rigorously tested in the context of medical image registration. This paper aims to fill that gap by examining this assumption for Gaussian Process (GP) registration.

Methods

The core of the paper involves evaluating the relationship between registration uncertainty and registration error using two main approaches:

  • Gaussian Process (GP) Registration: The authors utilize a GP-based approach for non-rigid registration. This method models displacement vectors as a Gaussian Process, allowing it to estimate both the predicted displacement and its associated uncertainty. The uncertainty is derived from the posterior covariance matrix of the GP.
  • Spearman's Rank Correlation Coefficient ($\rho_s$): To quantify the monotonic relationship between uncertainty and error, they employ Spearman's rank correlation. This non-parametric measure is chosen over Pearson's correlation because it assesses monotonic relationships (not necessarily linear) and is less sensitive to outliers, which are common in medical imaging data.
  • Point-wise Experiment: For each voxel (or landmark), they compute the registration error ($\epsilon$) and the GP-derived uncertainty ($\mu$). They then calculate $\rho_s$ for these paired values across the dataset.
  • Patch-wise Experiment: To account for the fact that surgeons are often interested in specific regions (e.g., tumor margins), they also analyze the correlation over image patches. For patches, they use intensity-based dissimilarity metrics like Histogram Intersection (HI) and Sum of Squared Differences (SSD) as proxies for patch-wise registration error.

Datasets

The study uses two clinical datasets for neurosurgical registration:

  • RESECT: A publicly available benchmark dataset for IGN.
  • MIBS: A proprietary dataset from a local hospital.

Both datasets consist of pre-operative (p-US) and intra-operative (i-US) ultrasound scans from patients with brain tumors. For RESECT, manually annotated landmarks were used. For MIBS, automatically detected landmarks were employed, as manual annotations were not available.

Key Insights and Ideas

  • Intuition vs. Reality: The common clinical intuition that high registration uncertainty directly implies high registration error is challenged. While there's a positive correlation, it's not strong enough to be a perfect predictor.
  • GP Uncertainty as a "Weak" Surrogate: GP registration uncertainty can provide some indication of error, but it's not a definitive measure. This means surgeons should be cautious about solely relying on uncertainty maps to dismiss potentially misregistered areas.
  • Impact of Deformation and Data: The study suggests that severe tissue deformation (like tumor resection) and the nature of landmark placement (e.g., distance to interpolating points in GP) can influence the uncertainty-error relationship.
  • Patch-wise vs. Point-wise: The patch-wise analysis yielded even lower correlation coefficients than the point-wise analysis. This highlights that local intensity-based dissimilarity metrics might not be ideal surrogates for error in highly deformed regions, and that the definition of "error" at the patch level needs careful consideration.

Key Results

  • Point-wise Correlation: The estimated Spearman's rank correlation coefficients ($\rho_s$) for the point-wise experiment were:
  • RESECT (manual landmarks): Average $\rho_s = 0.2899$ (weak-to-moderate positive correlation).
  • MIBS (automatic landmarks): Average $\rho_s = 0.4014$ (moderate-to-strong correlation).
  • Patch-wise Correlation: The estimated $\rho_s$ for the patch-wise experiment (using SSD/HI with patch sizes 3x3x3 and 5x5x5) were consistently low across both datasets.
  • Overall Finding: The study concludes that there is a weak-to-moderate positive monotonic correlation between GP registration uncertainty and non-rigid registration error. This suggests that current GP uncertainty measures may not be sufficient to reliably assess registration accuracy in practice.

Limitations and Future Work

  • Limitations:
    • The study is described as a "pilot study," indicating that further validation is needed.
  • The correlation coefficients, while positive, are not high enough to confidently use uncertainty as a direct proxy for error in clinical settings.
  • The patch-wise error estimation using intensity-based metrics might be suboptimal in regions with significant deformation.
  • The authors acknowledge that "label uncertainty" (related to intensity values) might be inherently uncorrelated with "transformation uncertainty" (related to displacement).
  • Future Work:
    • The paper opens a research area for uncertainty/error relationship analysis in medical image registration.
  • Further research is needed to verify and enhance the applicability of registration uncertainty.
  • Exploring alternative methods for estimating registration error, especially in highly deformed regions, is suggested.
  • Investigating the impact of different GP kernels and regularization parameters on the uncertainty-error relationship could be beneficial.
  • The authors suggest that estimating regional $\rho_s$ might be more informative than global measures.

Summary updated

Abstract

In image-guided neurosurgery, current commercial systems usually provide only rigid registration, partly because it is harder to predict, validate and understand non-rigid registration error. For instance, when surgeons see a discrepancy in aligned image features, they may not be able to distinguish between registration error and actual tissue deformation caused by tumor resection. In this case, the spatial distribution of registration error could help them make more informed decisions, e.g., ignoring the registration where the estimated error is high. However, error estimates are difficult to acquire. Probabilistic image registration (PIR) methods provide measures of registration uncertainty, which could be a surrogate for assessing the registration error. It is intuitive and believed by many clinicians that high uncertainty indicates a large error. However, the monotonic association between uncertainty and error has not been examined in image registration literature. In this pilot study, we attempt to address this fundamental problem by looking at one PIR method, the Gaussian process (GP) registration. We systematically investigate the relation between GP uncertainty and error based on clinical data and show empirically that there is a weak-to-moderate positive monotonic correlation between point-wise GP registration uncertainty and non-rigid registration error.
BibTeX
@inproceedings{luo2019are,
  author = {Jie Luo and Sarah F. Frisken and Duo Wang and Alexandra J. Golby and Masashi Sugiyama and William M. Wells III},
  title = {{Are Registration Uncertainty and Error Monotonically Associated?}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2019},
  url = {https://openreview.net/forum?id=Pla0cvZbetT},
}
Figure