Data-efficient Large Scale Place Recognition with Graded Similarity Supervision

Maria Leyva-Vallina, Nicola Strisciuglio, Nicolai Petkov

Abstract

Visual Place Recognition (VPR) is a critical component of autonomous navigation, yet traditional training methods suffer from computational inefficiencies and label noise. Standard approaches rely on binary labels—classifying image pairs as either identical or different—which necessitates expensive "hard-pair mining" to achieve convergence. This paper introduces a novel approach that utilizes graded similarity supervision based on camera pose metadata. By calculating the Field of View (FoV) overlap between images, we assign continuous similarity scores that more accurately reflect the visual relationship between scenes. We propose the Generalized Contrastive Loss (GCL) to leverage these scores, enabling models to converge up to 100x faster than traditional methods while achieving state-of-the-art performance without the need for complex mining strategies.

Introduction

Visual Place Recognition (VPR) involves identifying the geographic location of a query image by matching it against a database of known locations. Traditionally, this is treated as a metric learning problem where the goal is to map images into a latent space where descriptors of the same location are close and descriptors of different locations are far apart.

However, the standard binary labeling system (0 for different, 1 for same) introduces significant "label noise." For instance, in many datasets, images within 25 meters are considered a match. This creates a sharp, arbitrary boundary: an image at 24 meters is a positive match, while one at 26 meters is a negative, even if their visual content is nearly identical. To overcome the resulting optimization challenges, researchers typically employ hard-pair mining—a process of searching through massive datasets for the most informative examples. This process is computationally expensive and significantly slows down the training cycle.

Methodology

Graded Similarity Re-labeling

To move beyond binary constraints, we propose a supervision signal based on continuous similarity scores ($\psi$). Instead of a hard threshold, we calculate the visual overlap between two images using available metadata. For outdoor environments, we model the camera as a 2D sector using GPS and compass data, calculating the Intersection-over-Union (IoU) of these sectors. For indoor environments, we utilize 3D point cloud overlap. This provides a proxy for visual similarity that ranges from 0 (no overlap) to 1 (identical view).

Generalized Contrastive Loss (GCL)

We introduce the Generalized Contrastive Loss (GCL) to incorporate these continuous values into the training process. Unlike standard contrastive loss, GCL modulates the gradient signal based on the degree of similarity. When two images share only a partial overlap, the network is not forced to produce identical descriptors; instead, it pushes them together only as much as their actual visual overlap warrants. This naturally incorporates "soft negatives" into the training process, eliminating the need for hard-pair mining and allowing the model to learn a more nuanced and better-organized latent space.

Experimental Results

Training Efficiency

The most striking result of this approach is the improvement in training speed. A VGG-16 backbone trained with GCL reached peak performance approximately 100x faster than the industry-standard NetVLAD. Furthermore, the VGG16-GeM-GCL model requires 6x less memory, making large-scale VPR training accessible on more modest hardware.

Accuracy and Generalization

We evaluated our models on the Mapillary Street Level Sequences (MSLS) dataset, as well as several out-of-distribution sets including Pittsburgh30k, Tokyo24/7, and RobotCar Seasons v2.

  • State-of-the-Art Performance: Our ResNeXt-GeM-GCL model achieved a Top-5 Recall of 76.2% on the MSLS test set, outperforming more complex transformer-based architectures like TransVPR (74%) without requiring expensive re-ranking steps.
  • Robustness: The models demonstrated excellent generalization. On the Tokyo24/7 dataset, applying PCA whitening to our descriptors significantly boosted performance, outperforming many models specifically tuned for those environments.

Discussion and Limitations

Our findings suggest that the quality of the supervision signal is often more important than the complexity of the model architecture. By providing higher-quality labels, simpler backbones (like ResNet or ResNeXt with GeM pooling) can outperform more sophisticated models.

However, the method is dependent on the availability of accurate camera pose metadata (GPS and compass) or 3D models to generate graded labels. While such metadata is standard in modern research datasets, it may not be available for all "in-the-wild" web imagery. Additionally, FoV overlap is a 2D proxy and does not account for physical occlusions, such as walls, that might exist between two physically close cameras.

Conclusion

By replacing binary labels with graded similarity scores and utilizing the Generalized Contrastive Loss, we have demonstrated a path toward more data-efficient and computationally accessible Visual Place Recognition. This approach not only accelerates training by two orders of magnitude but also improves the robustness and accuracy of the resulting descriptors, paving the way for the development of larger foundation models for autonomous navigation.

Summary updated

Abstract

Visual place recognition (VPR) is a fundamental task of computer vision for visual localization. Existing methods are trained using image pairs that either depict the same place or not. Such a binary indication does not consider continuous relations of similarity between images of the same place taken from different positions, determined by the continuous nature of camera pose. The binary similarity induces a noisy supervision signal into the training of VPR methods, which stall in local minima and require expensive hard mining algorithms to guarantee convergence. Motivated by the fact that two images of the same place only partially share visual cues due to camera pose differences, we deploy an automatic re-annotation strategy to re-label VPR datasets. We compute graded similarity labels for image pairs based on available localization metadata. Furthermore, we propose a new Generalized Contrastive Loss (GCL) that uses graded similarity labels for training contrastive networks. We demonstrate that the use of the new labels and GCL allow to dispense from hard-pair mining, and to train image descriptors that perform better in VPR by nearest neighbor search, obtaining superior or comparable results than methods that require expensive hard-pair mining and re-ranking techniques. Code and models available at: https://github.com/marialeyvallina/generalized_contrastive_loss
BibTeX
@inproceedings{leyvavallina2023data,
  author = {Maria Leyva-Vallina and Nicola Strisciuglio and Nicolai Petkov},
  title = {{Data-efficient Large Scale Place Recognition with Graded Similarity Supervision}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  url = {https://openreview.net/forum?id=t8oqvcrUg9},
}
Figure