GeLoc3r: Enhancing Relative Camera Pose Regression with Geometric Consistency Regularization

Jingxing Li, Yongjae Lee, Deliang FanSubmission to ICLR 2026 · Desk rejected

TL;DR

GeLoc3r breaks the longstanding trade-off between the fast inference of camera pose regression and the high precision of correspondence solvers. By introducing Geometric Consistency Regularization (GCR)—a training-only framework that leverages privileged ground-truth depth and a learned weighted RANSAC solver—it teaches a regression network to understand 3D geometry. At test time, all geometric solver modules are discarded, preserving a blazing 33 ms inference speed while delivering state-of-the-art regression accuracy.

What Problem Does This Paper Address?

Estimating the relative camera pose between two images typically forces practitioners to choose between two paradigms:

  1. Correspondence-based matching methods (e.g., DUSt3R, MASt3R, ROMA): Highly accurate because they explicitly solve geometric equations (like PnP or epipolar constraints), but slow (300–400+ ms per pair).
  2. Direct pose regression methods (e.g., ReLoc3R): Fast (25 ms) feed-forward neural networks, but limited by an accuracy ceiling.

Prior research hypothesized that direct regression struggles because networks simply memorize appearance mappings rather than understanding 3D geometry. The authors discover that even high-performing regression models suffer from severe internal descriptor inconsistency: projecting pixels across views using the predicted pose maps to completely incorrect visual features. Hybrid inference methods (like FAR) fix this by running both regression and geometric solvers together at test time, but this destroys the speed advantage of regression.

Methods

GeLoc3r resolves this dilemma by making geometric verification a training-only supervisory signal. The inference pipeline remains a standard, fast Vision Transformer regression head.

Training:   Images ──> ViT Backbone ──> [Pose Head] ─── Pregression ──┐ Consistency
                           │                                         ├── Loss
                           └──> [Descriptor Heads] ──> FusionTransformer ──> Weighted RANSAC ──> Psolver
Inference:  Images ──> ViT Backbone ──> [Pose Head] ─── Pregression (33ms)

The training framework incorporates two novel supervisory mechanisms:

  • Learned Correspondence Weighting: Putative descriptor correspondences are formed using frozen pre-trained MASt3R features. A compact, 2-block transformer (FusionTransformer) applies self-attention across these correspondence pairs to learn reliability weights $w$, effectively modeling multi-correspondence consensus to filter out occlusions and textureless regions.
  • Direct & Indirect Geometric Supervision:
  • Direct (Pose Consistency): Ground-truth depth unprojects image points to 3D. A weighted RANSAC solver samples minimal subsets according to the learned weights $w$, solves PnP, and scores hypotheses using both geometric inliers and a soft prior favoring the current regression pose ($P_{\text{regression}}$). An angular consistency loss penalizes discrepancies between the regression pose and the solved pose ($P_{\text{solver}}$).
  • Indirect (Projection Verification): A projection descriptor loss ensures that 3D points projected via the predicted pose align with high cosine similarity against their ground-truth target descriptors.

Datasets

  • Training: GeLoc3r is trained on 10 diverse RGB-D datasets providing ground-truth depth: CO3Dv2, ScanNet++, ARKitScenes, MegaDepth, Waymo Open, StaticThings3D, Habitat, BlendedMVS, RealEstate10K, and WildRGB-D (50,000 pairs sampled per dataset per epoch).
  • Evaluation:
  • Relative Pose Estimation: MegaDepth1500, RealEstate10K, ScanNet1500, CO3Dv2, and BlendedMVS.
  • Visual Localization (Zero-shot / Unseen): Evaluated without fine-tuning on indoor 7-Scenes and outdoor Cambridge Landmarks.
  • Note: No new datasets were introduced.

Key Insights and Ideas

  • Privileged Training Information: Geometric constraints usually require depth, which is unavailable at inference. However, large-scale training datasets do have depth. GeLoc3r uses depth strictly as privileged training information to build a self-reinforcing curriculum.
  • Decoupled Architecture vs. Training Signal: Previous attempts to bridge the regression gap modified inference architectures. GeLoc3r demonstrates that the weakness of pose regression is not architectural capacity, but training loss design—minimizing $\lVert P_{\text{pred}} - P_{\text{gt}} \rVert$ alone fails to teach cross-view descriptor agreement.
  • Contextual Inlier Weighting via Self-Attention: Rather than evaluating match confidence independently, the FusionTransformer models inter-correspondence context, discovering consistent geometric patterns before RANSAC even samples.

Key Results

  • Superior Regression Accuracy: Across all relative pose benchmarks, GeLoc3r sets a new state of the art for pose regression:
  • CO3Dv2: Reaches 40.45% AUC@5° compared to ReLoc3R’s 34.85% (a 16% relative gain).
  • RealEstate10K: Increases AUC@5° from 66.70% to 68.66%.
  • MegaDepth1500: Improves AUC@20° under extreme viewpoint shifts to 82.55% (vs. 81.2% for ReLoc3R).
  • Trajectory Robustness: On the outdoor Cambridge Landmarks GreatCourt sequence (612 test frames), GeLoc3r mitigates catastrophic outlier errors, reducing trajectory RMSE by 41% (6.411 m vs. 10.862 m) and maximum localization error by 58%.
  • Efficiency: Runs in 33 ms on an NVIDIA RTX 4090 GPU—roughly 10× to 60× faster than correspondence pipelines like MASt3R (294 ms) or ROMA (300 ms).

Limitations and Future Work

  • Dependence on Ground-Truth Depth during Training: GCR requires depth maps and camera intrinsics during training, restricting its application to datasets with 3D annotations or high-fidelity sensor depth.
  • Two-View Scope: The current formulation is restricted to pairwise relative camera poses. Extending GCR to multi-view joint optimization and global structure-from-motion is left to future work.
  • Indoor Generalization Plateau: While GeLoc3r improved rotation on individual scenes, its overall median metric error on indoor 7-Scenes (0.04 m / 1.09°) was essentially on par with ReLoc3R (0.04 m / 1.02°), indicating diminishing returns in small, texture-rich indoor rooms.

Summary updated

Abstract

Prior ReLoc3R achieves breakthrough performance with fast 25ms inference and state-of-the-art regression accuracy, yet our analysis reveals subtle geometric inconsistencies in its internal representations that prevent reaching the precision ceiling of correspondence-based methods like MASt3R (which require 300ms per pair). In this work, we present GeLoc3r, a novel approach to relative camera pose estimation that enhances pose regression methods through Geometric Consistency Regularization (GCR). GeLoc3r overcomes the speed-accuracy dilemma by training regression networks to produce geometrically consistent poses without inference-time geometric computation. During training, GeLoc3r leverages ground-truth depth to generate dense 3D-2D correspondences, weights them using a FusionTransformer that learns correspondence importance, and computes geometrically-consistent poses via weighted RANSAC. This creates a consistency loss that transfers geometric knowledge into the regression network. Unlike FAR method which requires both regression and geometric solving at inference, GeLoc3r only uses the enhanced regression head at test time, maintaining ReLoc3R's fast speed and approaching MASt3R's high accuracy. On challenging benchmarks, GeLoc3r consistently outperforms ReLoc3R, achieving significant improvements including 40.45% vs. 34.85% AUC@5° on the CO3Dv2 dataset (16% relative improvement), 68.66% vs. 66.70% AUC@5° on RealEstate10K, and 50.45% vs. 49.60% on MegaDepth1500. By teaching geometric consistency during training rather than enforcing it at inference, GeLoc3r represents a paradigm shift in how neural networks learn camera geometry, achieving both the speed of regression and the geometric understanding of correspondence methods.
BibTeX
@misc{li2025geloc,
  author = {Jingxing Li and Yongjae Lee and Deliang Fan},
  title = {{GeLoc3r: Enhancing Relative Camera Pose Regression with Geometric Consistency Regularization}},
  year = {2025},
  eprint = {2509.23038},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.23038},
}

Loading related content...

Figure