Image registration using the shearlet transform

Glenn R. Easley, Monica Barbu-McInnis, Demetrio LabateProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2015 · DOI

TL;DR

This paper introduces a novel image registration algorithm that leverages the shearlet transform to extract robust and invariant features. By creating scale, rotation, and shift-invariant key feature vectors, their method achieves accurate alignment of images, demonstrating strong performance on medical imaging datasets.

What Problem Does This Paper Address?

Image registration is a fundamental task in many fields, including medical imaging and remote sensing, where the goal is to align two or more images of the same scene. This is crucial for tasks like combining information from different viewpoints, monitoring changes over time, or fusing data from different sensors. The core challenge in image registration lies in accurately identifying corresponding points or features between images, especially when they are subject to variations in viewpoint, scale, rotation, and illumination. Traditional methods often rely on feature detection, matching, transformation estimation, and resampling, but finding reliable and invariant features is a persistent hurdle.

Methods

The paper proposes an image registration algorithm built upon the shearlet transform. Shearlets are a powerful mathematical tool that, unlike traditional wavelets, can capture multi-scale and multi-directional information with high anisotropy. This makes them particularly adept at representing geometric features like edges and corners in images.

The key innovation lies in using the shearlet representation to extract Shearlet Extracted Robust Features (SERF). The process involves:

  1. Feature Detection: Local extrema are identified in the shearlet domain. This is achieved by applying directional filtering (using shearlets) and a Laplacian Pyramid (LP) to the image. The LP helps in scale-invariant detection.
  2. Descriptor Generation: For each detected keypoint, a descriptor is generated. This involves calculating the scattering transform (a related multi-scale representation) of the local image patch around the keypoint. The scattering transform provides invariance to translation and small deformations. The paper specifically mentions using scattering transform values up to order 2, resulting in a descriptor vector of length 96.
  3. Feature Matching: The generated SERF descriptors from two images are matched using the Mahalanobis distance, with matches kept below a threshold.
  4. Transformation Estimation: Once correspondences are established, a standard RANSAC algorithm is used to estimate the transformation (rigid or non-rigid) between the images.

The authors highlight that their SERF descriptors are a refinement of existing methods like SURF, offering improved invariance and robustness.

Datasets

The paper demonstrates its registration algorithm on various medical databases. Specifically, they mention using two-dimensional image slices extracted from MRI image stacks, obtained from references 20 and 21. No new benchmarks or datasets were introduced.

Key Insights and Ideas

  • Shearlets for Feature Extraction: The paper effectively demonstrates that the multi-scale and multi-directional properties of shearlets are highly beneficial for detecting salient image features that are crucial for registration.
  • SERF as a Refinement: The proposed SERF features, built upon shearlets and scattering transforms, are presented as a more robust and invariant alternative to existing local descriptors like SURF. This is achieved by combining the strengths of shearlets for feature localization and scattering transforms for descriptor robustness.
  • Invariance is Key: The core idea is to create feature vectors that are invariant to scale, rotation, and illumination changes, which are common challenges in image registration. This invariance is critical for reliable matching between images acquired under different conditions.

Key Results

The paper presents qualitative results demonstrating the effectiveness of the SERF features for image registration.

  • Accurate Matching: Figures 3, 4, 5, and 6 visually show that SERF descriptors successfully match corresponding points between images, even when subjected to rigid transformations (Figures 4 and 5) and non-rigid deformations (Figure 6).
  • Comparison to SURF: Figure 3 directly compares SERF with SURF (using a Laplacian Pyramid implementation), suggesting that SERF can achieve more accurate correspondences.
  • Robustness: The algorithm is shown to be effective on MRI image stacks, indicating its utility in challenging medical imaging scenarios.

Quantitative results (e.g., accuracy metrics, speedup) are not explicitly detailed in the provided text.

Limitations and Future Work

  • Limitations:
    • The paper focuses on 2D image registration.
  • While qualitative results are shown, quantitative performance metrics are not detailed.
  • The paper mentions that the shearlet construction is "essentially written in space-domain as (1)" and that their supports are "contained inside the trapezoidal regions," implying potential complexities or approximations in the practical implementation.
  • Future Work:
    • The authors plan to explore the applicability of their approach by incorporating additional feature elements, such as directed graphs.
  • They also aim to extend the method to higher dimensions.

Summary updated

Abstract

In this paper, we present an algorithm for image registration utilizing the shearlet representation. The shearlet framework allows one to collect multi-scale and multi-directional feature information from multidimensional data that can be used to create key feature vectors that are scale, rotation, and shift invariant. These key feature vectors produce a transformation that will align the sensed image to the source image. We demonstrate our registration algorithm on various medical databases.
BibTeX
@inproceedings{easley2015image,
  author = {Glenn R. Easley and Monica Barbu-McInnis and Demetrio Labate},
  title = {{Image registration using the shearlet transform}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2015},
  doi = {10.1117/12.2188816},
}
Figure