Abstract
Absolute Pose Regression (APR) has emerged as a compelling paradigm for visual localization. However, APR models typically operate as black boxes, directly regressing a 6-DoF pose from a query image, which can lead to memorizing training views rather than understanding 3D scene geometry. In this work, we propose a geometrically-grounded alternative. Inspired by novel view synthesis, which renders images from intermediate geometric representations, we reformulate APR as its inverse that regresses the underlying 3D representations directly from the image, and we name this paradigm Geometric Representation Regression (GRR). Our model explicitly predicts two disentangled geometric representations in the world coordinate system: (1) a ray bundle's directions to estimate camera rotation, and (2) a corresponding pointmap to estimate camera translation. The final 6-DoF camera pose is then recovered from these geometric components using a differentiable deterministic solver. This disentangled approach, which separates the learned visual-to-geometry mapping from the final pose calculation, introduces a strong geometric prior into the network. We find that the explicit decoupling of rotation and translation predictions measurably boosts performance. We demonstrate state-of-the-art performance on 7-Scenes and Cambridge Landmarks datasets, validating that modeling the inverse rendering process is a more robust path toward generalizable absolute pose estimation.
Introduction
Visual localization—determining a camera's exact 6-DoF pose from a single image—is fundamental to robotics and augmented reality. While traditional methods rely on complex feature-matching pipelines, Absolute Pose Regression (APR) has gained popularity for its simplicity, using a single neural network to map images directly to pose vectors.
However, APR often suffers from a "memorization" problem. Instead of learning 3D scene geometry, these models can act as retrieval systems, matching query images to training views. This leads to poor generalization. GRLoc addresses this by introducing Geometric Representation Regression (GRR), forcing the network to predict intermediate 3D geometric structures—rays and pointmaps—rather than raw coordinates.
Methods
The GRLoc architecture moves away from black-box regression by framing localization as the inverse of Novel View Synthesis (NVS). If NVS renders pixels from geometry, GRR recovers geometry from pixels.
Dual-Branch Architecture
GRLoc utilizes a Swin Transformer backbone that splits into two specialized heads:
- Ray Branch: Predicts a "ray bundle" (unit vectors) in world space, used specifically for estimating camera rotation.
- Point Branch: Predicts a "pointmap" (3D coordinates) in world space, used for estimating camera translation.
This decoupling is crucial because rotation and translation are often competing objectives in a single network. By splitting them, the model avoids interference between the two tasks.
Differentiable Deterministic Solver
Instead of the network guessing the final pose, the predicted rays and points are processed by a differentiable Kabsch algorithm. This classical mathematical tool finds the optimal alignment between the predicted geometry and the camera's local coordinate system. Being differentiable, it allows for end-to-end training while providing a strong geometric prior.
Data Augmentation via 3DGS
To overcome data scarcity, the authors employ 3D Gaussian Splatting (3DGS) to generate synthetic training views. A Gradient Reversal Layer (GRL) is used for domain adaptation, ensuring the model learns features that generalize across both real and synthetic images.
Key Results
GRLoc was evaluated on the 7-Scenes and Cambridge Landmarks datasets, setting new benchmarks for APR methods.
On Cambridge Landmarks, GRLoc achieved an average reduction of over 33% in translation error and 46% in rotation error compared to previous state-of-the-art APR methods. On 7-Scenes, it outperformed specialized models like Marepo, with median errors as low as 0.97cm in specific scenes. When further refined (GRLoc$_{ref}$), the system's accuracy becomes competitive with high-cost Scene Coordinate Regression (SCR) methods.
Conclusion and Future Work
GRLoc demonstrates that incorporating explicit geometric priors into absolute pose regression significantly improves localization accuracy and robustness. By decoupling rotation and translation and utilizing a differentiable solver, the model moves closer to a true understanding of 3D scene geometry.
Future work includes improving the model's dependency on high-quality 3DGS reconstructions, which can be challenging in very large or sparse outdoor environments. Additionally, exploring more efficient architectures could reduce the computational overhead of the dual-branch transformer design.
Summary updated