Relative Pose Estimation of Calibrated Cameras with Known SE(3) Invariants

TL;DR

This paper presents a comprehensive study of relative pose estimation for calibrated cameras, introducing novel methods that leverage known SE(3) invariants (rotation angle and screw translation) to reduce the number of required image points. This leads to more efficient and robust solutions, particularly for applications involving robots with known motion constraints.

What Problem Does This Paper Address?

Estimating the relative pose (rotation and translation) between two cameras is a fundamental problem in 3D computer vision, crucial for applications like robot localization, mapping, and augmented reality. Traditional methods, such as the 5-point algorithm, require a minimum number of point correspondences. While incorporating additional constraints can reduce this number, these methods often rely on known extrinsic calibration to transform external information (like gravity direction from an IMU) into the camera's coordinate frame. This paper addresses the challenge of efficiently and robustly estimating relative camera pose when certain geometric invariants of the rigid motion (SE(3) invariants) are known, without requiring explicit extrinsic calibration.

Methods

The core contribution of this work is a systematic exploration of how SE(3) invariants – specifically the rotation angle ($\theta$) and screw translation ($\delta$) – can be used to constrain the relative pose estimation problem. The authors break down the problem into five minimal cases based on which invariants are known ($\delta=0$, $\theta$ known, or both known).

Key methodological aspects include:

  • SE(3) Invariant Formulation: The paper formally defines and proves the invariance of the rotation angle and screw translation under the "hand-eye transformation" (conjugation by an SE(3) element), demonstrating why extrinsic calibration is not needed.
  • Minimal Problem Reduction: By incorporating these invariants as constraints, the number of required 2D-3D point correspondences is reduced. For instance, knowing $\delta=0$ (planar motion) can reduce the problem from 5 points to 3 points.
  • Comprehensive Study of Polynomial Formulations: The authors conduct a thorough review and analysis of existing polynomial formulations for relative pose estimation (e.g., SIR2, SIR3, SIR6, NullE). They analyze their pros and cons, reveal connections between them, and select the most efficient formulation for each of the proposed minimal problems.
  • Hidden Variable Method: For solving the resulting polynomial systems, the paper utilizes the hidden variable method, which reduces the problem to finding real roots of a univariate polynomial, enhancing computational efficiency.

Datasets

The paper evaluates its proposed methods on both synthetic and real-world datasets.

  • Synthetic Data: Generated with varying levels of Gaussian image noise (0-1px) and different motion types (forward and sideway). The rotation angle measurement was also disturbed by Gaussian noise. For non-perfectly planar motion, translation was disturbed along the rotation axis.
  • Real-World Data: Datasets collected from indoor mobile robots and outdoor autonomous vehicles. Specific datasets mentioned include:
  • RawSeeds-Bicocca
    • TUM-RGBD-SLAM
    • Robot@Home
    • KITTI
    • UMich
    • Robot Car

Key Insights and Ideas

  • SE(3) Invariants as Powerful Constraints: The paper highlights that known SE(3) invariants are highly effective constraints for reducing the complexity of relative pose estimation.
  • Eliminating Extrinsic Calibration: A major advantage is the ability to use these invariants without needing to calibrate the camera's extrinsics relative to other sensors (like IMUs) or motion planes, making the methods more flexible and convenient.
  • Unified Framework for Minimal Problems: By systematically studying all combinations of known SE(3) invariants, the paper provides a unified view of minimal relative pose problems, revealing relationships between previously disparate approaches.
  • Formulation Matters for Efficiency: The study emphasizes that the choice of polynomial formulation (e.g., SIR2, SIR3, NullE) significantly impacts the efficiency and template size for solving these minimal problems.

Key Results

  • Performance Improvement: Experiments on synthetic and real data demonstrate performance improvements compared to conventional relative pose estimation methods (e.g., 5P).
  • Reduced Point Requirements: The proposed methods, by leveraging SE(3) invariants, can solve relative pose problems with fewer points (e.g., 3-point solutions for planar motion).
  • Robustness: The paper shows that 4P-ST0 (a 4-point solver with zero screw translation constraint) generally has more real roots than the 5P solver, which can be beneficial in RANSAC frameworks.
  • Specific Solver Strengths:
    • 4P-ST0 shows significant advantages in translation error for forward motion.
  • 3P-RA-ST0 is more sensitive to screw translation error in forward motion compared to 4P-ST0 but can outperform 5P under certain conditions.
  • On urban narrow road environments, the 5P solver sometimes achieves better accuracy than 4P-ST0, suggesting limitations of the planar motion assumption in such scenarios.

Limitations and Future Work

  • Degeneracy Handling: The paper acknowledges that the SE(3) invariant condition $\delta=0$ becomes degenerate when the rotation matrix is close to the identity matrix. This degenerate case is handled by fitting relative pose to a translation-only motion (2P-TO) and accepting results with more inliers.
  • Environmental Sensitivity: The performance of methods relying on planar motion assumptions (like 4P-ST0) can degrade in environments where the "planar" assumption is less accurate (e.g., urban roads with tilting).
  • Future Work: The paper's comprehensive study and formulation of minimal problems with SE(3) invariants lay the groundwork for developing even more efficient solvers. The authors' focus on practical applications and experimental validation suggests future work could involve further integration into real-time robotic systems and exploration of more complex generalized camera models.

Summary updated

BibTeX
@inproceedings{2020relative,
  title = {{Relative Pose Estimation of Calibrated Cameras with Known SE(3) Invariants}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
}
Figure