UniPR-3D: Towards Universal Visual Place Recognition with Visual Geometry Grounded Transformer

Tianchen Deng, Xun Chen, Ziming Li, Hongming Shen, Danwei Wang, Javier Civera, Hesheng WangarXiv 2025 · DOI · arXiv

Abstract

Visual Place Recognition (VPR) is a critical task for autonomous navigation, yet it remains challenging due to environmental variations and limited perspective. This paper presents UniPR-3D, a novel framework that integrates 3D geometric information into the VPR pipeline using a Visual Geometry Grounded Transformer (VGGT). By combining 2D texture-based features with 3D structural tokens, UniPR-3D achieves state-of-the-art performance across diverse benchmarks, demonstrating superior robustness to seasonal changes and viewpoint shifts.

1. Introduction

Visual Place Recognition (VPR) serves as the fundamental "localization" component in robotic navigation and Simultaneous Localization and Mapping (SLAM). The task involves identifying a previously visited location by matching a query image against a database of geo-tagged images. Traditional VPR methods primarily rely on 2D texture-based features. However, these methods often fail in dynamic environments where lighting, weather, or seasonal changes drastically alter the visual appearance of a scene. Furthermore, single-view approaches often lack the spatial context necessary to disambiguate similar-looking locations.

To address these challenges, we introduce UniPR-3D. Unlike conventional models that treat images as flat 2D grids, UniPR-3D leverages 3D spatial information to ground visual features in a geometric context. This approach provides a more stable representation of the environment, as structural geometry is significantly more invariant to appearance changes than surface textures.

2. Methodology

The UniPR-3D framework is built upon three core innovations: the VGGT backbone, a specialized feature aggregation strategy, and a flexible sequence matching mechanism.

2.1 The Visual Geometry Grounded Transformer (VGGT)

At the heart of UniPR-3D is the VGGT backbone. While standard Vision Transformers (ViTs) process images as independent patches, VGGT utilizes "alternating attention" blocks to ground features in 3D space. The architecture extracts two distinct types of information:

  • 2D Tokens: Extracted via a DINOv2 encoder, these tokens capture high-resolution texture and semantic details (e.g., the presence of a specific sign or vehicle).
  • 3D Tokens: Produced by the VGGT blocks, these encode the underlying 3D structural layout of the scene (e.g., the orientation of walls and the depth of the street canyon).

2.2 Feature Aggregation and Global Descriptors

To create a robust global descriptor, UniPR-3D employs a dual-stream aggregation strategy tailored to the nature of the tokens:

  • Generalized Mean (GeM) Pooling: Applied to global [CLS] and register tokens to capture dominant semantic cues.
  • Optimal Transport (Sinkhorn): Inspired by the SALAD architecture, this module processes patch tokens by computing a soft matching plan. This preserves structural correspondences across different views, enhancing robustness against viewpoint shifts.

2.3 Variable-Length Sequence Matching

UniPR-3D extends beyond single-frame retrieval by incorporating a multi-frame projector. The system defines an "anchor frame" and a set of "support frames." A key advantage of this design is its support for variable-length sequences. The model can perform retrieval using an arbitrary number of frames at inference time (e.g., 3, 5, or 15), regardless of the sequence length used during training.

3. Experimental Evaluation

The model was trained on GSV-Cities and Mapillary Street-Level Sequences (MSLS) and evaluated across several challenging benchmarks, including Pittsburgh250k, NordLand, Oxford Robotcar, and SPED.

3.1 Performance Metrics

UniPR-3D establishes a new state-of-the-art (SOTA) across multiple datasets:

  • MSLS Challenge: Achieved 74.3% Recall@1 (R@1), outperforming the previous SOTA, SALAD (73.0%).
  • Oxford Robotcar: Reached 95.4% R@1, a significant improvement over CaseVPR (90.5%).
  • NordLand: In the extreme Summer-vs-Winter matching task, UniPR-3D achieved 76.2% R@1, nearly 7% higher than the DINOv2-based baseline.

4. Discussion

Analysis of the model's attention mechanisms reveals that 2D features typically focus on transient objects and surface details, whereas 3D tokens focus on permanent structural elements like building edges and walls. This "geometric anchoring" is the primary reason for the model's success in cross-season and day-night matching. Furthermore, the model demonstrates a positive scaling property: as the number of frames in a sequence increases, the retrieval accuracy improves consistently without the need for retraining.

5. Limitations and Conclusion

While UniPR-3D provides superior accuracy, it introduces higher computational overhead. The latency is approximately 8.23ms per frame, compared to 2.41ms for SALAD. Future work will focus on optimizing the 3D backbone for real-time deployment on resource-constrained robotic platforms and exploring the integration of 360-degree panoramic inputs.

In conclusion, UniPR-3D demonstrates that grounding visual features in 3D geometry is a powerful paradigm for universal visual place recognition, offering unprecedented robustness in complex, real-world environments.

Summary updated

Abstract

Visual Place Recognition (VPR) has been traditionally formulated as a single-image retrieval task. Using multiple views offers clear advantages, yet this setting remains relatively underexplored and existing methods often struggle to generalize across diverse environments. In this work we introduce UniPR-3D, the first VPR architecture that effectively integrates information from multiple views. UniPR-3D builds on a VGGT backbone capable of encoding multi-view 3D representations, which we adapt by designing feature aggregators and fine-tune for the place recognition task. To construct our descriptor, we jointly leverage the 3D tokens and intermediate 2D tokens produced by VGGT. Based on their distinct characteristics, we design dedicated aggregation modules for 2D and 3D features, allowing our descriptor to capture fine-grained texture cues while also reasoning across viewpoints. To further enhance generalization, we incorporate both single- and multi-frame aggregation schemes, along with a variable-length sequence retrieval strategy. Our experiments show that UniPR-3D sets a new state of the art, outperforming both single- and multi-view baselines and highlighting the effectiveness of geometry-grounded tokens for VPR. Our code and models will be made publicly available on Github https://github.com/dtc111111/UniPR-3D.
BibTeX
@misc{deng2025unipr,
  author = {Tianchen Deng and Xun Chen and Ziming Li and Hongming Shen and Danwei Wang and Javier Civera and Hesheng Wang},
  title = {{UniPR-3D: Towards Universal Visual Place Recognition with Visual Geometry Grounded Transformer}},
  year = {2025},
  eprint = {2512.21078},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.21078},
}
Figure