Homography from two orientation- and scale-covariant features

Daniel Barath, Zuzana Kukelova

Abstract

This paper proposes a geometric interpretation of the angles and scales which the orientation- and scale-covariant feature detectors, e.g. SIFT, provide. Two new general constraints are derived on the scales and rotations which can be used in any geometric model estimation tasks. Using these formulas, two new constraints on homography estimation are introduced. Exploiting the derived equations, a solver for estimating the homography from the minimal number of two correspondences is proposed. Also, it is shown how the normalization of the point correspondences affects the rotation and scale parameters, thus achieving numerically stable results. Due to requiring merely two feature pairs, robust estimators, e.g. RANSAC, do significantly fewer iterations than by using the four-point algorithm. When using covariant features, e.g. SIFT, the information about the scale and orientation is given at no cost. The proposed homography estimation method is tested in a synthetic environment and on publicly available real-world datasets.

Introduction

Estimating a homography—a transformation mapping points between two planes—is a cornerstone of computer vision, essential for image stitching, planar object tracking, and 3D reconstruction. Traditionally, this task requires a minimum of four point correspondences. However, modern feature detectors like SIFT and SURF provide more than just spatial coordinates; they also output a local scale and a dominant orientation.

While these extra parameters are typically used only as descriptors for matching, this paper introduces a mathematically rigorous framework to treat them as geometric constraints. By unlocking this "hidden" information, the authors propose a solver that requires only two correspondences, leading to a 10–100x speedup in robust estimation frameworks like RANSAC.

Geometric Interpretation and Methodology

The core contribution is a new geometric interpretation of how local feature scales and rotations relate to the global homography matrix ($H$). The authors decompose the affine transformation into a product of Jacobians ($A = J_2 J_1^{-1}$), proving that the uniform scale $q_i$ reported by detectors is exactly the determinant of the local Jacobian of the projection function.

Geometric interpretation of scale ($q$) and orientation ($\alpha$) for a feature point $P$ projected onto two image planes.

To handle unknowns like local surface shear and non-uniform scaling, the authors employed elimination ideal theory from algebraic geometry. This allowed them to derive two new constraints independent of these unknowns, resulting in a "minimal solver" that uses just two feature pairs. This solver efficiently handles a system of two quadratic equations, yielding up to four potential homography solutions while maintaining numerical stability through a novel normalization formula for affine parameters.

Experimental Results

The 2-SIFT solver was evaluated against standard benchmarks, including the AdelaideRMF, Multi-H, Malaga, and Strecha datasets. The results demonstrate that moving from a four-point to a two-point minimal set significantly reduces the computational burden of RANSAC. For instance, on the AdelaideRMF dataset, the 2-SIFT solver required an average of only 877 iterations, compared to 26,082 for the standard four-point algorithm.

Comparison of transfer error distributions for the proposed 2SIFT solver versus the standard 4-point (4PT) and 3-point ORB (3ORB) methods.

Crucially, this speedup does not come at the cost of accuracy. The re-projection error remained comparable to the four-point method, with the 2-SIFT method achieving a mean error of 1.57 px versus 1.61 px for the traditional approach on the AdelaideRMF dataset. The solver also proved robust against noise in the feature's orientation and scale.

Conclusion and Future Work

By mathematically formalizing the geometric meaning of covariant feature parameters, this work enables more efficient planar transformation estimation. While the method's precision is tied to the accuracy of the underlying feature detector's scale and orientation estimates, the massive reduction in RANSAC iterations makes it highly practical for real-time applications. Future research may extend these general constraints to other geometric problems, such as estimating essential or fundamental matrices with fewer correspondences.

Summary updated

Abstract

This paper proposes a geometric interpretation of the angles and scales which the orientation- and scale-covariant feature detectors, e.g. SIFT, provide. Two new general constraints are derived on the scales and rotations which can be used in any geometric model estimation tasks. Using these formulas, two new constraints on homography estimation are introduced. Exploiting the derived equations, a solver for estimating the homography from the minimal number of two correspondences is proposed. Also, it is shown how the normalization of the point correspondences affects the rotation and scale parameters, thus achieving numerically stable results. Due to requiring merely two feature pairs, robust estimators, e.g. RANSAC, do significantly fewer iterations than by using the four-point algorithm. When using covariant features, e.g. SIFT, the information about the scale and orientation is given at no cost. The proposed homography estimation method is tested in a synthetic environment and on publicly available real-world datasets.
BibTeX
@inproceedings{barath2019homography,
  author = {Daniel Barath and Zuzana Kukelova},
  title = {{Homography from two orientation- and scale-covariant features}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=KAuylHNQAJH},
}

Loading related content...

Figure