Abstract
Visual relocalization is a fundamental capability for autonomous robots, enabling them to recover their 6-degree-of-freedom (6-DoF) pose within a known environment. This paper introduces a novel framework that utilizes pre-built 3D surfel (surface element) maps to aid visual relocalization. By combining learned descriptors with a surfel-based geometric optimization, the system achieves high accuracy and robustness while being significantly more efficient to build than traditional Structure-from-Motion models. Evaluation on real-world and simulated datasets demonstrates that the approach maintains high recall and metric accuracy, making it well-suited for real-time robotic applications.
Introduction
Relocalization allows a robot to determine its position when it loses track or starts in a previously mapped area. While visual SLAM is widely used, it often suffers from cumulative drift. High-accuracy alternatives like Structure-from-Motion (SfM) provide precise maps but are computationally expensive and difficult to update. Conversely, 3D Lidar maps provide excellent geometry but require expensive sensors. This work bridges the gap by using a high-quality 3D surfel map—often readily available from Lidar-equipped mapping platforms—to guide a low-cost camera-based relocalization system.
Methodology
The proposed system follows a three-stage pipeline designed for efficiency and precision.
1. Visual Database Construction
Instead of a full SfM reconstruction, the system renders a pre-existing 3D surfel map from known camera poses. This process directly associates 2D image pixels with 3D surfels, creating a visual database where every feature is grounded in the environment's geometry. This approach is roughly 20 times faster than SfM-based methods like COLMAP.
2. Surfel Reprojection Optimization
A key innovation is the use of surfel properties, such as surface normals, to constrain map points. Unlike traditional methods that treat map points as independent dots, this approach defines a cost function that minimizes reprojection error by forcing points to lie on the planes defined by the surfels. This geometric constraint significantly improves the consistency of the visual database.
3. Hierarchical Relocalization
When processing a query image, the system employs a two-stage hierarchical approach:
- Global Retrieval: NetVLAD descriptors are used to identify the most similar database images.
- Local Matching: Learned descriptors (e.g., SuperPoint) or traditional features (ORB) establish precise 2D-3D correspondences. The final pose is estimated via a PnP solver and refined through motion-only optimization.
Experimental Results
The system was evaluated using the EuRoC MAV dataset (indoor UAV flights) and the CARLA Simulator (outdoor autonomous driving).
Performance and Accuracy
In the EuRoC dataset, the SuperPoint-based configuration achieved a relocalization recall of 93.5%, with a Mean Absolute Trajectory Error (mATE) as low as 2.82 cm. In outdoor simulations, the surfel optimization proved critical for maintaining accuracy even when initial database poses were noisy.
Efficiency
The method is highly practical for deployment. The relocalization itself takes less than 50ms per frame, supporting real-time operation on standard robotic hardware.
Discussion and Future Work
The study highlights that learned descriptors like SuperPoint offer superior robustness to lighting and viewpoint changes compared to traditional ORB features. However, the system currently assumes a static environment. Future research will focus on online map updates to accommodate dynamic scenes where the physical environment changes over time. Additionally, while the relocalizing robot only needs a camera, the initial map generation still relies on a high-end mapping setup, typically involving Lidar.
Summary updated