Image matching with scale adjustment

Yves Dufournaud, Cordelia Schmid, Radu Horaud

Abstract

In this paper we address the problem of matching two images with two different resolutions: a high-resolution image and a low-resolution one. The difference in resolution between the two images is not known and without loss of generality one of the images is assumed to be the high-resolution one. On the premise that changes in resolution act as a smoothing equivalent to changes in scale, a scale-space representation of the high-resolution image is produced. Hence the one-to-one classical image matching paradigm becomes one-to-many because the low-resolution image is compared with all the scale-space representations of the high-resolution one. Key to the success of such a process is the proper representation of the features to be matched in scale-space. We show how to represent and extract interest points at variable scales and we devise a method allowing the comparison of two images at two different resolutions. The method comprises the use of photometric- and rotation-invariant descriptors, a geometric model mapping the high-resolution image onto a low-resolution image region, and an image matching strategy based on local constraints and on the robust estimation of this geometric model. Extensive experiments show that our matching method can be used for scale changes up to a factor of 6.

Introduction

Automated image matching is a fundamental task in computer vision, yet it remains challenging when images are captured at significantly different resolutions. Standard interest point detectors, such as the Harris corner detector, are typically designed for one-to-one matching between images of similar scale. When a high-resolution "zoom" image must be registered against a low-resolution "context" image, these standard tools often fail because features change appearance or disappear entirely under different levels of blur and sampling.

This research introduces a "one-to-many" matching framework. By treating resolution differences as equivalent to Gaussian smoothing, we can model the relationship between images using scale-space theory. This allows us to match a single low-resolution image against a stack of blurred versions of the high-resolution image, identifying the optimal scale for registration.

Methodology

Scale-Space Representation and Feature Adaptation

The core innovation lies in simulating the loss of resolution through Gaussian convolution. We generate a scale-space for the high-resolution image by applying kernels with increasing standard deviations ($s$). To ensure that interest points remain detectable and stable across these scales, we adapt the Harris detector. Specifically, we derive a scaling law showing that the cornerness threshold must be adjusted by $s^4$ to maintain consistency across the scale-space.

Scale-space representation showing interest point extraction at different scales ($s=1, 3, 5, 7$).

Matching and Robust Estimation

The matching pipeline utilizes differential invariants to achieve rotation invariance. The process involves initial matching using Mahalanobis distance between descriptors, followed by the enforcement of local geometric constraints within neighborhoods of points. Finally, RANSAC is used to robustly estimate a similitude transformation (rotation, translation, and scale) to finalize the registration.

Experimental Results

The proposed method was tested on diverse real-world datasets, most notably mountain scenes captured with a 6x zoom factor (using 12mm and 72mm lenses).

Matching results on a wide-angle mountain scene, showing the identified region and interest points.

Our experiments demonstrate that the scale-adapted Harris detector maintains a repeatability rate of nearly 80-100% at a scale factor of 6, whereas standard detectors drop below 40% repeatability beyond a scale factor of 2. The robust matching strategy proved capable of identifying the correct transformation even in the presence of high outlier ratios at non-optimal scales.

Conclusion

We have presented a method to bridge the resolution gap in image matching by moving to a one-to-many scale-space paradigm. By adapting interest point detection with an $s^4$ scaling law, we achieve robust registration for scale changes up to a factor of 6. Future work will focus on relaxing the planar scene assumption and improving computational efficiency to handle more complex 3D environments.

Summary updated

Abstract

In this paper we address the problem of matching two images with two different resolutions: a high-resolution image and a low-resolution one. The difference in resolution between the two images is not known and without loss of generality one of the images is assumed to be the high-resolution one. On the premise that changes in resolution act as a smoothing equivalent to changes in scale, a scale-space representation of the high-resolution image is produced. Hence the one-to-one classical image matching paradigm becomes one-to-many because the low-resolution image is compared with all the scale-space representations of the high-resolution one. Key to the success of such a process is the proper representation of the features to be matched in scale-space. We show how to represent and extract interest points at variable scales and we devise a method allowing the comparison of two images at two different resolutions. The method comprises the use of photometric- and rotation-invariant descriptors, a geometric model mapping the high-resolution image onto a low-resolution image region, and an image matching strategy based on local constraints and on the robust estimation of this geometric model. Extensive experiments show that our matching method can be used for scale changes up to a factor of 6.
BibTeX
@article{dufournaud2004image,
  author = {Yves Dufournaud and Cordelia Schmid and Radu Horaud},
  title = {{Image matching with scale adjustment}},
  journal = {Comput. Vis. Image Underst.},
  year = {2004},
  url = {https://openreview.net/forum?id=WLHZc1d5XD},
}
Figure