Abstract
Visual place recognition is a challenging task in computer vision and a key component of camera-based localization and navigation systems. Recently, Convolutional Neural Networks (CNNs) achieved high results and good generalization capabilities. They are usually trained using pairs or triplets of images labeled as either similar or dissimilar, in a binary fashion. In practice, the similarity between two images is not binary, but continuous. Furthermore, training these CNNs is computationally complex and involves costly pair and triplet mining strategies. We propose a Generalized Contrastive loss (GCL) function that relies on image similarity as a continuous measure, and use it to train a siamese CNN. Furthermore, we present three techniques for automatic annotation of image pairs with labels indicating their degree of similarity, and deploy them to re-annotate the MSLS, TB-Places, and 7Scenes datasets. We demonstrate that siamese CNNs trained using the GCL function and the improved annotations consistently outperform their binary counterparts. Our models trained on MSLS outperform the state-of-the-art methods, including NetVLAD, NetVLAD-SARE, AP-GeM and Patch-NetVLAD, and generalize well on the Pittsburgh30k, Tokyo 24/7, RobotCar Seasons v2 and Extended CMU Seasons datasets. Furthermore, training a siamese network using the GCL function does not require complex pair mining.
Introduction: The Problem of Binary Similarity
Visual Place Recognition (VPR) is the task of identifying whether a query image depicts a location already stored in a database. Most deep learning approaches treat this as a binary classification problem: two images are either "similar" (same place) or "dissimilar" (different place).
However, similarity is naturally continuous. Two images might share 80% of a scene, 20%, or nothing at all. Forcing these into binary categories creates training inconsistencies. For example, if image A overlaps with B, and B overlaps with C, a binary system might label both pairs as "similar." By the transitive property, the network tries to push A and C together in the embedding space, even if A and C share zero visual features. This "label noise" hinders the network's ability to learn a robust representation of space.
Methodology
The core contribution is the Generalized Contrastive Loss (GCL). Unlike standard contrastive loss, which uses a binary label $y \in \{0, 1\}$, GCL uses a continuous similarity score $\psi \in [0, 1]$.
The GCL Function
The loss function is designed to optimize the distance between image representations in a latent space proportionally to their annotated degree of similarity. This acts as an intrinsic regularizer, ensuring the learned embedding space reflects the actual geometry of the environment. Images are positioned relative to one another based on physical overlap rather than just being pushed together or pulled apart.
Automatic Continuous Labeling
To train with GCL, the authors propose three automated techniques to re-annotate existing datasets:
- Weak 2D Field-of-View (FoV) overlap: Uses GPS and compass data to estimate how much two camera frustums intersect on a 2D plane.
- Strong 2D FoV overlap: Uses precise 6DOF camera poses for better accuracy.
- 3D FoV overlap: Uses 3D reconstructions and camera poses to calculate the intersection-over-union (IoU) of visible 3D points.
Simplified Training
Because the loss function is more informative, the authors can use a "naïve mining" strategy. Instead of complex, memory-intensive algorithms that search for "hard" examples during training, they simply ensure each batch has a roughly uniform distribution of similarity grades.
Key Insights: Addressing Logical Inconsistency
A major insight of this work is the resolution of binary inconsistency. Binary labels force networks to learn conflicting representations when partial overlaps occur. This is conceptually illustrated with geometric shapes: if A is similar to B, and B is similar to C, a binary system incorrectly implies A is similar to C, even if they share no features.
Results and Generalization
The authors re-annotated several major VPR datasets (MSLS, TB-Places, 7Scenes) and evaluated generalization on benchmarks like Pittsburgh30k and Tokyo 24/7.
- State-of-the-Art Performance: On the MSLS test set, their ResNeXt-GeM-GCL model achieved a Top-5 recall of 76.2%, outperforming existing SOTA methods like NetVLAD (44%), NetVLAD-SARE, and Patch-NetVLAD.
- Consistency: Across all tested backbones (VGG16, ResNet50, ResNet152, ResNeXt), models trained with GCL consistently outperformed those trained with standard binary contrastive loss.
- Generalization: Models trained on MSLS using GCL generalized remarkably well to unseen datasets. For instance, on Tokyo 24/7, they achieved an 81% Top-5 recall, significantly better than the 67% achieved by standard NetVLAD.
Conclusion
By treating image similarity as a continuous spectrum, the Generalized Contrastive Loss provides a more robust and physically grounded objective for training Siamese networks. This approach not only achieves state-of-the-art results in visual place recognition but also simplifies the training process by eliminating the need for complex hard-pair mining. Future work includes integrating GCL into full SLAM pipelines for more robust navigation.
Summary updated