PUMP: Pyramidal and Uniqueness Matching Priors for Unsupervised Learning of Local Descriptors

Jerome Revaud, Vincent Leroy, Philippe Weinzaepfel, Boris Chidlovskii

Abstract

Existing approaches for learning local image descriptors have shown remarkable achievements in a wide range of geometric tasks. However, most of them require per-pixel correspondence-level supervision, which is difficult to acquire at scale and in high quality. In this paper, we propose to explicitly integrate two matching priors in a single loss in order to learn local descriptors without supervision. Given two images depicting the same scene, we extract pixel descriptors and build a correlation volume. The first prior enforces the local consistency of matches in this volume via a pyramidal structure iteratively constructed using a nonparametric module. The second prior exploits the fact that each descriptor should match with at most one descriptor from the other image. We combine our unsupervised loss with a standard self-supervised loss trained from synthetic image augmentations. Feature descriptors learned by the proposed approach outperform their fully- and selfsupervised counterparts on various geometric benchmarks such as visual localization and image matching, achieving state-of-the-art performance. Project webpage: https: //europe.naverlabs.com/research/3d-vision/pump.
BibTeX
@inproceedings{revaud2022pump,
  author = {Jerome Revaud and Vincent Leroy and Philippe Weinzaepfel and Boris Chidlovskii},
  title = {{PUMP: Pyramidal and Uniqueness Matching Priors for Unsupervised Learning of Local Descriptors}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.00390},
}

Loading related content...

Figure