Novel OCT Mosaicking Pipeline with Feature-And Pixel-Based Registration

Jiacheng Wang, Hao Li, Dewei Hu, Yuankai K. Tao, Ipek Oguz

TL;DR

This paper introduces a novel, robust pipeline for creating high-resolution, wide-field-of-view OCT images by stitching together multiple smaller scans. It cleverly combines advanced deep learning for initial alignment with traditional pixel-based methods for fine-tuning, and uniquely uses the Segment Anything Model (SAM) for unsupervised quality assessment.

What Problem Does This Paper Address?

Optical Coherence Tomography (OCT) is a powerful imaging technique for ophthalmology, but individual OCT scans have a limited field of view (FoV). To get a broader picture of the retina, researchers stitch together multiple overlapping OCT images – a process called mosaicking. However, existing mosaicking methods often struggle with OCT images that have significant noise, deformation, or only slight overlaps between scans. These issues can lead to inaccurate alignments and "ghosting" artifacts, compromising the diagnostic value of the final mosaic.

Methods

The proposed pipeline adopts a four-stage, coarse-to-fine strategy:

  1. Coarse Stage: Feature-based Alignment: This stage uses learning-based feature detection and matching. Specifically, it employs SuperPoint for robust keypoint detection and LightGlue for matching these features. LightGlue is highlighted for its improved accuracy and efficiency over previous methods like SuperGlue.
  2. Bridge Stage: Affine Registration for Features: Instead of traditional methods that can be sensitive to initial conditions or cause distortions, this stage creates "feature images" where matching feature pairs are assigned unique intensities. This allows for a direct intensity-based affine image registration using a global correlation metric, which is more robust to noisy or isolated features.
  3. Refinement Stage: Pixel-based Deformable Registration: Building on the coarse affine alignment, this stage uses the ANTs SyN algorithm for deformable registration. This fine-tunes local variations and corrects for non-rigid deformations, ensuring precise pixel-level alignment.
  4. Verification Stage: Feature Prompts for SAM Segmentation: A novel unsupervised evaluation method is introduced. It leverages the Segment Anything Model (SAM). Feature points identified in Stage I (specifically, those matched by LightGlue as positive prompts and low-confidence features as negative prompts) are used to prompt SAM. The consistency of SAM's segmentation of the mosaic versus the reference image in overlapping regions serves as a quality metric, indicating how well vessel structures are preserved.

Datasets

  • In-house 9-field dataset: Acquired using a Spectrally Encoded Coherence Tomography and Reflectometry (SECTR) system. This dataset consists of high-resolution OCT sub-field volumes from two healthy subjects, processed to simulate noise and deformation.
  • Synthetic public OCTA-500 dataset: A publicly available dataset of human OCT en-face projections. The authors artificially created sub-fields and introduced random affine transformations and elastic displacements to simulate mosaicking conditions, providing ground truth for quantitative evaluation.

Key Insights and Ideas

  • Bridging Feature and Pixel-Based Methods: The pipeline effectively combines the global robustness of learning-based feature matching with the local precision of pixel-based deformable registration, addressing the limitations of each approach individually.
  • Novel Affine Registration Strategy: Using feature intensity maps for affine registration offers a more stable and less distortion-prone alternative to traditional methods like RANSAC for initial alignment.
  • Unsupervised Evaluation with SAM: The innovative use of SAM with feature prompts provides a powerful way to assess mosaicking quality, especially in medical contexts where dense ground truth is often unavailable. This opens new avenues for evaluating and comparing mosaicking algorithms.
  • LightGlue's Efficacy: The paper highlights LightGlue as a highly accurate and efficient feature matching method, outperforming traditional and other learning-based approaches for OCT image registration.

Key Results

  • The proposed Affine+SyN method significantly outperformed the MAGSAC baseline in quantitative comparisons on the 9-field dataset across three metrics: Mean Square Error (MSE), Structural Similarity Index Measure (SSIM), and Peak Signal to Noise Ratio (PSNR). Improvements were statistically significant.
  • Qualitative comparisons showed that the proposed method produced mosaics with fewer "ghost" vessels and better-aligned structures compared to the MAGSAC baseline, even after refinement.
  • The SAM-based unsupervised evaluation demonstrated that the refinement stage significantly improved mosaicking outcomes, with the proposed method achieving the highest Dice score (0.75) in segmentation agreement.
  • LightGlue was found to be highly accurate and efficient for feature matching, balancing performance with computational time.

Limitations and Future Work

  • Unsupervised Evaluation Limitations: While SAM provides a novel evaluation method, the paper suggests that future work could further enhance this by incorporating other segmentation comparison metrics like average surface distance (ASD) and 95-percent Hausdorff distance.
  • Generalizability: While validated on in-house and synthetic datasets, further testing on a wider variety of real-world OCT datasets with diverse pathologies and imaging conditions would be beneficial.
  • Computational Efficiency: Although LightGlue is noted for its efficiency, the overall pipeline's computational cost for large datasets might still be a consideration for real-time applications.

Summary updated

Abstract

High-resolution Optical Coherence Tomography (OCT) images are crucial for ophthalmology studies but are limited by their relatively narrow field of view (FoV). Image mosaicking is a technique for aligning multiple overlapping images to obtain a larger FoV. Current mosaicking pipelines often struggle with substantial noise and considerable displacement between the input sub-fields. In this paper, we propose a versatile pipeline for stitching multi-view OCT/OCTA en face projection images. Our method combines the strengths of learning-based feature matching and robust pixel-based registration to align multiple images effectively. Furthermore, we advance the application of a trained foundational model, Segment Anything Model (SAM), to validate mosaicking results in an unsupervised manner. The efficacy of our pipeline is validated using an in-house dataset and a large public dataset, where our method shows superior performance in terms of both accuracy and computational efficiency. We also made our evaluation tool for image mosaicking and the corresponding pipeline publicly available at https://github.com/MedICL-VU/OCT-mosaicking.
BibTeX
@misc{wang2023novel,
  author = {Jiacheng Wang and Hao Li and Dewei Hu and Yuankai K. Tao and Ipek Oguz},
  title = {{Novel OCT Mosaicking Pipeline with Feature-And Pixel-Based Registration}},
  year = {2023},
  eprint = {2311.13052},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.13052},
}
Figure