Revisiting Global Translation Estimation with Feature Tracks

Peilin Tao, Hainan Cui, Mengqi Rong, Shuhan ShenCVPR 2024 · DOI

Abstract: Global Structure from Motion (SfM) is an efficient alternative to incremental methods but often fails in scenarios with collinear camera motion, such as autonomous driving. We introduce Hybrid Explicit Translation Averaging (HETA), a robust framework that combines relative translations with high-quality feature tracks. HETA utilizes a parallax-aware weighting scheme and a two-stage optimization process to overcome degeneracy. Our evaluation on the KITTI and 1DSfM datasets demonstrates that HETA significantly outperforms existing global SfM methods in accuracy and robustness, particularly in challenging collinear motion sequences.

Introduction

Structure from Motion (SfM) is a fundamental task in computer vision, aimed at reconstructing 3D scenes from 2D image sequences. While incremental SfM is widely used for its robustness, it suffers from high computational costs and cumulative drift. Global SfM methods estimate all camera poses simultaneously, offering better scalability. However, global translation estimation remains a major challenge. Most methods rely on relative translations, which become degenerate when cameras move in a straight line—a common occurrence in automotive datasets like KITTI.

Methodology

HETA addresses these challenges through a hybrid explicit approach that integrates camera-to-camera and camera-to-point constraints.

1. Parallax-Aware Re-estimation

Before global optimization, HETA refines relative translations. By utilizing the coplanarity constraint without normalizing the epipolar plane normal vectors, the method naturally weights matches by their parallax. High-parallax matches, which provide more stable geometric information, are prioritized, while low-parallax matches are down-weighted.

2. Feature Track Selection

To maintain efficiency and reduce noise, HETA selects a subset of high-quality feature tracks. Tracks are ranked by their maximum parallax, ensuring that the chosen tracks provide strong geometric constraints across the camera sequence.

Geometric representation of feature tracks connecting multiple camera centers ($t_l, t_i, t_r$) to a 3D point $P_k$.

3. Two-Stage Optimization

The framework employs a robust two-step process. First, an $L_1$ norm optimization is used to handle outliers effectively. This is followed by an $L_2$ norm refinement using Iteratively Reweighted Least Squares (IRLS) to achieve high precision.

Experimental Results

The performance of HETA was evaluated on the KITTI Odometry benchmark and the 1DSfM dataset.

KITTI Superiority

On the KITTI dataset, characterized by collinear motion, HETA achieved a median error of approximately 0.1 meters. In contrast, baseline global methods like LUD and LiGT often failed to produce meaningful trajectories or exhibited errors in the tens of meters.

Comparison of estimated trajectories on KITTI sequences 02, 05, and 06. The top row shows HETA's close alignment with Ground Truth, while the bottom row illustrates the failure of baseline methods (LUD, CReTA, LiGT, PGILP) under collinear motion.

Internet Image Robustness

On unordered internet images (1DSfM), HETA demonstrated superior robustness by registering more images than competing methods while maintaining high accuracy, proving its effectiveness in handling noisy, large-scale data.

Conclusion

HETA provides a robust solution to the long-standing problem of translation estimation in global SfM. By leveraging feature tracks and parallax-aware weighting, it overcomes the limitations of collinear motion. Future work includes integrating deep learning-based feature matching to further enhance the reliability of the initial view graph construction.

Summary updated

BibTeX
@inproceedings{tao2024revisiting,
  author = {Peilin Tao and Hainan Cui and Mengqi Rong and Shuhan Shen},
  title = {{Revisiting Global Translation Estimation with Feature Tracks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.01955},
}
Figure