Abstract
Visual localization is a fundamental challenge in robotics, typically requiring scene-specific retraining or extensive image databases. This paper proposes Scene-agnostic Pose Regression (SPR), a paradigm shift that enables robots to localize in unseen environments by predicting poses relative to a trajectory's origin. We introduce SPR-Mamba, a dual-branch architecture utilizing State Space Models for long-range dependency modeling, and 360SPR, a large-scale panoramic dataset. Our approach achieves drift-free localization in unknown scenes, significantly outperforming traditional absolute and relative pose regression methods.
Introduction
Visual localization—the process of determining a device's position and orientation from images—is critical for autonomous navigation. Existing methods generally fall into three categories, each with significant drawbacks:
- Absolute Pose Regression (APR): Requires retraining for every new scene.
- Relative Pose Regression (RPR): Generalizes better but depends on a large, computationally expensive image "atlas."
- Visual Odometry (VO): Works anywhere but suffers from "drift," where errors accumulate over time.
To resolve this "localization dilemma," the authors introduce the SPR task. Instead of mapping to a global coordinate system, the model uses the first image of a sequence ($I_1$) as the origin and predicts the 6D pose of subsequent query images ($I_q$) relative to that origin. This allows the model to learn the geometry of movement rather than specific landmark appearances.
Methodology: SPR-Mamba
The proposed SPR-Mamba model is designed to handle long trajectories with high efficiency. It consists of several key components:
- Feature Extraction: A frozen DINO (Vision Transformer) backbone extracts robust visual features.
- Local Branch: Captures short-term, frame-to-frame motion cues.
- Global Branch: Employs Mamba (State Space Models) to aggregate information across entire sequences. Unlike standard Transformers, Mamba maintains linear complexity relative to sequence length, making it ideal for the long-range dependencies found in robotic paths.
- Fusion and Prediction: The model fuses local and global information to regress the final translation and rotation. By anchoring predictions to the initial frame, the global branch effectively eliminates the cumulative drift common in VO.
The 360SPR Dataset
A significant contribution of this work is the creation of the 360SPR dataset, designed to address the limitations of existing benchmarks.
- Scale: It contains over 200,000 photorealistic panoramas and 3.6 million pinhole images across 270 diverse indoor environments.
- Multi-Height Sensing: To ensure the model works across different robotic platforms, data was collected at three distinct heights: 0.1m (e.g., vacuum robots), 0.5m (e.g., quadrupeds), and 1.7m (e.g., humanoids).
The use of 360° panoramas is central to the SPR task, as the wider field of view provides the necessary visual overlap to maintain stable relative pose estimation across varying viewpoints.
Experimental Results
SPR-Mamba demonstrates state-of-the-art performance in zero-shot generalization to unknown scenes. In unseen environments, SPR-Mamba reduced translation and rotation errors by over 23m and 43° compared to APR, and 7m and 16° compared to RPR.
On the 360Loc benchmark, the model achieved a median error of 1.94m and 1.44° in entirely new environments, proving its efficacy on real photographic data. Ablation studies confirmed that the Global Branch is the "secret sauce"; without it, the model loses its anchor to the starting point, leading to significant performance degradation over long trajectories.
Conclusion and Future Work
SPR-Mamba represents a significant step toward general-purpose robot navigation. By disentangling scene appearance from motion geometry, the researchers enable localization in novel environments without the need for maps or retraining. Future work will focus on handling dynamic obstacles in crowded environments and explicitly modeling the distortions inherent in panoramic imagery to further refine precision.
Summary updated