Abstract
We propose a lightweight retrieval-based pipeline to predict 6DOF camera poses from RGB images. Our pipeline uses a convolutional network (CNN) to encode a query image as a feature vector. A nearest neighbor lookup finds the pose-wise nearest database image. A siamese convolutional neural network regresses the relative pose from the nearest neighboring database image to the query image. The relative pose is then applied to the nearest neighboring absolute pose to obtain the query image's final absolute pose prediction. Our model is a distilled version of NN-Net that reduces its parameters by 98.87%, information retrieval feature vector size by 87.5%, and inference time by 89.18% without a significant decrease in localization accuracy.
Introduction
Determining a camera's precise 6-degree-of-freedom (6DOF) pose from a single RGB image is a foundational task for autonomous systems and augmented reality. While traditional methods rely on matching 2D points to 3D maps, they are often computationally prohibitive for edge devices. Recent deep learning approaches have introduced retrieval-based methods like NN-Net, which find a similar image in a database and calculate a relative offset. However, these models remain bulky. DistillPose addresses this efficiency gap, providing a "distilled" model optimized for real-time performance on resource-constrained hardware.
Methodology
The DistillPose pipeline operates in two primary stages: Information Retrieval and Relative Pose Regression. First, a query image is encoded into a compact feature vector to find the closest match in a training database. Once retrieved, a siamese network calculates the relative pose difference between the query and the database image.
Layerwise Training and Auxiliary Learning
A key innovation in this work is the use of layerwise training. Rather than relying solely on the final layer of a ResNet34 backbone, the authors attached pose regression heads to each of the four residual blocks. During training, the model is further enhanced through auxiliary geometric tasks, such as predicting frustum overlap and angular distance, which forces the network to learn more robust geometric features.
Model Distillation
The final "DistillPose" model is a significantly pruned version of the full architecture. By identifying that earlier layers (L1 and L2) often capture more relevant geometric information for localization than deeper semantic layers, the authors were able to use only the first residual block for the final lightweight version.
Experimental Results
The model was evaluated on the Seven Scenes dataset, a standard benchmark for indoor relocalization. Results demonstrate that DistillPose achieves performance comparable to state-of-the-art models while being a fraction of the size:
- Efficiency Gains: Model size was reduced by 98.87% (from 22.3M to 248K parameters), and inference time dropped by 89.18% (from 37ms to 4ms).
- Storage: The retrieval feature vector size was reduced by 87.5%, significantly lowering database storage requirements.
- Accuracy: The best variant (L1 with auxiliary losses) achieved a median error of 0.23m and 9.85°, staying remarkably close to the much larger NN-Net (0.21m and 9.30°).
Discussion and Conclusion
DistillPose demonstrates that for camera localization, deeper architectures are not always superior. Mid-level geometric features from earlier CNN layers are highly effective for relative pose regression. By combining layerwise training with geometric auxiliary losses, the authors created a model capable of real-time 6DOF estimation on the edge. Future work will explore scaling this approach to larger outdoor environments and further optimizing backbones specifically for geometric tasks.
Summary updated