Introduction
Image matching is a fundamental task in computer vision, yet matching multimodal images—such as optical to infrared, MRI to PET, or optical to SAR—remains a significant challenge. These image pairs often exhibit Nonlinear Intensity Distortion (NID), where pixel values do not follow a simple linear relationship. Furthermore, significant differences in scale and arbitrary rotations between sensors frequently cause traditional gradient-based methods like SIFT to fail.
The NISR Framework
To address these challenges, the authors propose NISR (Nonlinear Intensity, Scale, and Rotation invariant matching). The framework leverages structural information rather than raw intensity values to achieve robustness across disparate sensor data.
Feature Detection and Orientation Estimation
The process begins with scale-space construction using an image pyramid to handle zooming differences. Features are detected using Phase Congruency (PC), which is inherently resistant to illumination changes because it operates in the frequency domain. A key innovation is the use of Orientation Index Maps (OIM). By convolving images with multi-scale, multi-orientated log-Gabor filters, each pixel is assigned an index corresponding to the filter with the maximum response.
To overcome rotation ambiguity, NISR employs a Double Index Map strategy, using two sets of filter layers (odd and even) to ensure distinct responses even at "half-step" rotation angles. Rotation invariance is achieved through a Weighted Centroid Orientation method, which calculates a stable primary orientation based on the structural landscape of the OIM.
Sparse-to-Dense Refinement
After initial sparse matching, the system performs a dense template match using 3D phase correlation. This transition from sparse features to a dense "cloud" of correspondences significantly improves registration precision and reliability.
Experimental Evaluation
The NISR algorithm was tested on a diverse dataset of 164 multimodal image pairs across computer vision, medical imaging, and remote sensing domains.
Performance Metrics
NISR demonstrated superior performance compared to state-of-the-art methods like RIFT and LNIFT:
- Success Rate: NISR achieved a 98.8% success rate, significantly outperforming RIFT (75.6%), which often failed on pairs with large rotations.
- Match Density: NISR found an average of 1,883 correct matches per pair, whereas competitors typically found fewer than 400.
- Precision: The Root Mean Square Error (RMSE) was reduced to 0.98 pixels, indicating high geometric accuracy.
Conclusion
NISR provides a robust solution for multimodal image matching by prioritizing structural consistency over intensity values. While the use of log-Gabor filters introduces computational overhead, the framework's resilience to NID, scale, and rotation makes it a powerful tool for complex registration tasks. Future work will focus on integrating deep learning to optimize the generation of orientation maps and improve real-time performance.
Summary updated