Abstract
This paper proposes a new image-based localization framework that explicitly localizes the camera/robot by fusing Convolutional Neural Network (CNN) and sequential images' geometric constraints. The camera is localized using a single or few observed images and training images with 6-degree-of-freedom pose labels. A Siamese network structure is adopted to train an image descriptor network, and the visually similar candidate image in the training set is retrieved to localize the testing image geometrically. Meanwhile, a probabilistic motion model predicts the pose based on a constant velocity assumption. The two estimated poses are finally fused using their uncertainties to yield an accurate pose prediction. This method leverages the geometric uncertainty and is applicable in indoor scenarios predominated by diffuse illumination. Experiments on simulation and real data sets demonstrate the efficiency of our proposed method. The results further show that combining the CNN-based framework with geometric constraint achieves better accuracy when compared with CNN-only methods, especially when the training data size is small.
Introduction
Indoor localization—determining a robot's or camera's exact 6-degree-of-freedom (6-DoF) pose—is a cornerstone of autonomous navigation. Current approaches generally fall into two categories: monocular SLAM and deep learning-based regression. While SLAM is effective for local tracking, it often suffers from scale drift and requires continuous image sequences. Conversely, deep learning models like PoseNet can map images directly to poses but act as "black boxes" requiring massive labeled datasets and struggling with novel viewing angles.
This research introduces a hybrid framework that combines deep learning for robust image retrieval with classical geometry for precise pose estimation. By focusing on similarity retrieval rather than direct coordinate regression, the system achieves high accuracy with significantly less training data.
Proposed Methodology
The framework integrates five core modules to provide a robust pose estimate by fusing visual descriptors with physical constraints.
1. Image Descriptor Network
A Siamese network architecture, based on GoogLeNet, is employed to learn robust image descriptors. Unlike regression-based networks, this model is trained using a contrastive loss to identify the most visually similar training image for any given test frame.
2. Geometric Locator and Keyframe Selection
Once a similar training image is retrieved, the system selects co-visible keyframes that share visual features. The Geometric Locator then utilizes forward-intersection to construct a local 3D map and backward-intersection to calculate the test camera's pose by minimizing re-projection errors.
3. Probabilistic Motion Model and Fusion
To mitigate the impact of noisy frames or motion blur, a probabilistic motion model predicts the camera's position based on a constant velocity assumption. The final pose is a weighted fusion of the geometric estimate and the motion model prediction, where weights are dynamically adjusted based on the calculated uncertainty (e.g., re-projection error) of each source.
Experimental Results
The method was evaluated using synthetic data from NVIDIA’s Isaac Sim and real-world benchmarks including Microsoft’s 7Scenes and the TUM RGB-D dataset.
- Superior Accuracy: On the TUM "desk2" sequence, the proposed method achieved a median error of 0.03m and 2.74°, significantly outperforming PoseNet (0.21m / 15.81°) and ContextualNet (0.05m / 10.75°).
- Robustness to Lighting: In simulated warehouse environments with challenging lighting, the fusion method maintained a median error of 0.51m, whereas PoseNet's error increased to 3.58m.
- Data Efficiency: The system demonstrated superior performance even when training data was reduced to 12% of the original set, showcasing its ability to generalize from sparse labels.
Conclusion
By leveraging geometry as a "sanity check" for neural network predictions, this framework provides a more reliable and data-efficient solution for indoor localization. Future work will focus on improving robustness to drastic illumination changes and extending the system to handle uncalibrated cameras with unknown intrinsic parameters.
Summary updated