Abstract
Relative pose estimation is a fundamental task in computer vision, typically requiring at least five point correspondences between image frames. This paper introduces a novel approach that leverages SE(3) invariants—specifically the rotation angle and screw translation—to develop minimal solvers that require as few as three points. Because these invariants are independent of the coordinate system on a rigid body, the proposed solvers can integrate auxiliary sensor data (e.g., from an IMU or wheel odometry) without requiring extrinsic calibration between the camera and the sensor. Experimental results demonstrate that these solvers are computationally efficient, robust to challenging motion patterns like forward translation, and offer significant accuracy improvements in real-world robotics and autonomous driving scenarios.
Introduction
In the field of computer vision and robotics, "relative pose estimation" involves determining the rotation and translation of a camera between two distinct frames. While the 5-point algorithm is the standard solution for calibrated cameras, it often struggles with noise and specific degenerate motions. Researchers frequently supplement camera data with other sensors, such as Inertial Measurement Units (IMUs), to simplify the problem. However, traditional integration methods require precise "extrinsic calibration"—knowing the exact spatial relationship between the camera and the sensor. If this calibration is inaccurate or changes due to mechanical stress, the system's performance degrades significantly.
This paper addresses this bottleneck by utilizing SE(3) invariants. In robotics, certain geometric properties of motion, such as the rotation angle and the screw translation (the component of movement along the axis of rotation), remain constant regardless of where the coordinate origin is placed on a rigid body. By focusing on these invariants, the authors enable "calibration-free" sensor fusion, allowing camera pose to be refined using external sensor data without knowing the camera's mounting position.
The Geometry of SE(3) Invariants
The core mathematical innovation lies in the proof that the rotation angle ($\theta$) and screw translation ($\delta$) are invariant under "hand-eye" transformations. This means that if a robot rotates by a certain angle, every sensor rigidly attached to it experiences that same rotation angle, even if their individual rotation matrices differ due to their orientations.
As shown in the figure above, the screw translation $\delta$ represents the displacement along the rotation axis $r$. For many common robotic platforms, such as ground vehicles moving on a plane, this screw translation is effectively zero ($\delta = 0$). By treating these values as known constraints derived from external sensors, the complexity of the relative pose problem is drastically reduced.
Proposed Minimal Solvers
The authors introduce a suite of five new minimal solvers tailored to different levels of available information. These solvers are categorized by the number of points (P) required and the specific invariants used as constraints.
Key configurations include:
- 4P-RA: Utilizes a known rotation angle, requiring only 4 point correspondences.
- 4P-ST0: Assumes zero screw translation (common in planar motion), requiring 4 points.
- 3P-RA-ST0: The most efficient solver, requiring only 3 points when both the rotation angle and the zero-screw-translation constraint are available.
- 5P-ST1 and 4P-RA-ST1: Specialized solvers for cases where screw translation is non-zero, allowing for the recovery of the absolute scale of movement.
Efficiency and Algebraic Formulations
A significant portion of the research focuses on the algebraic efficiency of these solvers. The authors conducted a comprehensive study of different polynomial formulations, including SIR2, SIR3, SIR6, and NullE. They discovered that the SIR2 formulation, which utilizes $2 \times 2$ determinants, is significantly more efficient than traditional methods. This choice leads to smaller "template matrices," enabling the solvers to execute in as little as 25 to 34 microseconds. This level of performance is critical for real-time applications in high-speed robotics.
Experimental Evaluation
The solvers were rigorously tested against synthetic data and several prominent real-world datasets, including RawSeeds-Bicocca, TUM-RGBD-SLAM, and the KITTI autonomous driving benchmark.
Key Findings:
- Robustness to Forward Motion: Standard algorithms often fail when a camera moves directly forward (the epipole problem). The 4P-ST0 and 3P-RA-ST0 solvers proved remarkably stable in these scenarios.
- Accuracy Gains: On the RawSeeds dataset, incorporating the rotation angle from an IMU via the 3P-RA-ST0 solver reduced the median translation error from 10.8° (using the standard 5-point algorithm) to just 5.6°.
- Planar Motion Flexibility: The solvers effectively handled autonomous driving data on highways where the planar motion assumption ($\delta = 0$) holds, even without explicit extrinsic calibration.
Conclusion and Future Work
By leveraging SE(3) invariants, this paper provides a robust framework for relative pose estimation that bypasses the need for tedious and error-prone extrinsic calibration. While the solvers show some sensitivity to "screw translation noise" in highly uneven terrain (like narrow urban streets with significant vehicle tilt), their speed and accuracy in standard navigation tasks are superior to existing methods. Future work will explore extending these invariant-based solvers to generalized camera systems and multi-camera rigs.
Summary updated