RIFT2: Speeding-up RIFT with A New Rotation-Invariance Technique

Jiayuan Li, Pengcheng Shi, Qingwu Hu, Yongjun ZhangarXiv 2023 · OpenReview

Abstract

Multimodal image matching is an important prerequisite for multisource image information fusion. Compared with the traditional matching problem, multimodal feature matching is more challenging due to the severe nonlinear radiation distortion (NRD). Radiation-variation insensitive feature transform (RIFT) has shown very good robustness to NRD and become a baseline method in multimodal feature matching. However, the high computational cost for rotation invariance largely limits its usage in practice. In this paper, we propose an improved RIFT method, called RIFT2. We develop a new rotation invariance technique based on dominant index value, which avoids the construction process of convolution sequence ring. Hence, it can speed up the running time and reduce the memory consumption of the original RIFT by almost 3 times in theory. Extensive experiments show that RIFT2 achieves similar matching performance to RIFT while being much faster and having less memory consumption. The source code is available at: https://github.com/LJY-RS/RIFT2-multimodal-matching-rotation

Introduction

Image matching across different modalities—such as Synthetic Aperture Radar (SAR), infrared, and optical sensors—is a fundamental task in remote sensing and computer vision. The primary obstacle is Nonlinear Radiation Distortion (NRD), where intensity values for the same scene vary significantly across sensors. While the Radiation-variation Insensitive Feature Transform (RIFT) algorithm effectively addresses NRD using Maximum Index Maps (MIM), its approach to rotation invariance is a major bottleneck. To handle unknown rotations, the original RIFT generates six redundant versions of every feature descriptor (a "convolution sequence ring"), leading to high computational and memory overhead.

Methodology: RIFT2

RIFT2 overhauls the rotation-invariance strategy of the original RIFT to improve efficiency without compromising robustness.

The RIFT2 framework overhauls the feature description stage to achieve rotation invariance more efficiently.

The core innovation lies in the Dominant Index Value (DIV) technique:

  1. Maximum Index Map (MIM) Construction: Log-Gabor filters are applied to the input images to create an MIM, which captures structural features resistant to radiation changes.
  2. Dominant Orientation Identification: Instead of brute-force matching against multiple rotated descriptors, RIFT2 analyzes the histogram of orientations within a local MIM patch to find the most frequent value—the "dominant index."
  3. Descriptor Normalization: The descriptor is "recoded" by shifting all orientation values relative to this dominant index. This aligns the descriptor to a canonical orientation, achieving mathematical rotation invariance with a single descriptor per point.
  4. Stability: To maintain robustness, if a secondary peak in the orientation histogram is significant (within 80% of the primary peak), an additional descriptor is generated, ensuring stability in complex scenes.

Experimental Results

The performance of RIFT2 was evaluated against the original RIFT and several SIFT variants (PSO-SIFT, OS-SIFT) using SAR-optical and infrared-optical datasets.

Quantitative comparison of RIFT2 against other state-of-the-art multimodal matching methods.

Efficiency and Accuracy

As shown in the experimental data, RIFT2 achieves a significant speedup. For SAR-optical matching, the running time dropped from 29.24 seconds (RIFT) to 10.93 seconds (RIFT2), confirming the theoretical ~3x improvement. Crucially, this speedup does not come at the cost of accuracy; RIFT2 maintained a 96-97% success rate, matching or slightly exceeding the original RIFT's performance.

Comparison with Traditional Methods

Traditional SIFT-based methods struggle with multimodal data, often achieving success rates as low as 2-3%. While specialized variants like OS-SIFT improve this to roughly 38% in some cases, they remain far behind the RIFT family in both success rate and the number of correct matches ($n$).

Conclusion

RIFT2 successfully addresses the computational bottleneck of the original RIFT algorithm. By replacing redundant descriptor generation with a histogram-based normalization technique, it achieves a ~3x speedup and reduced memory footprint. While currently optimized for similar-scale images, future work will focus on incorporating scale-space construction and GPU acceleration to move toward real-time multimodal image matching.

Summary updated

Abstract

Multimodal image matching is an important prerequisite for multisource image information fusion. Compared with the traditional matching problem, multimodal feature matching is more challenging due to the severe nonlinear radiation distortion (NRD). Radiation-variation insensitive feature transform (RIFT)~\cite{li2019rift} has shown very good robustness to NRD and become a baseline method in multimodal feature matching. However, the high computational cost for rotation invariance largely limits its usage in practice. In this paper, we propose an improved RIFT method, called RIFT2. We develop a new rotation invariance technique based on dominant index value, which avoids the construction process of convolution sequence ring. Hence, it can speed up the running time and reduce the memory consumption of the original RIFT by almost 3 times in theory. Extensive experiments show that RIFT2 achieves similar matching performance to RIFT while being much faster and having less memory consumption. The source code will be made publicly available in \url{https://github.com/LJY-RS/RIFT2-multimodal-matching-rotation}
BibTeX
@misc{li2023rift,
  author = {Jiayuan Li and Pengcheng Shi and Qingwu Hu and Yongjun Zhang},
  title = {{RIFT2: Speeding-up RIFT with A New Rotation-Invariance Technique}},
  year = {2023},
  eprint = {2303.00319},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.00319},
}
Figure