Abstract
Relative pose estimation is a cornerstone of 3D reconstruction and computer vision. Traditional methods, such as the 5-point algorithm, rely solely on 2D point correspondences. This paper introduces a novel polarimetric relative pose estimation framework that utilizes the surface normal information encoded in polarized light. By leveraging these additional geometric constraints, we derive a minimal 2-point solver that is more robust to noise and requires fewer correspondences than classical approaches. We further propose a joint optimization method to handle unknown material properties and orientation ambiguities. Our results on synthetic and real-world datasets demonstrate significant improvements in accuracy, particularly in challenging, noisy scenarios.
Introduction
Estimating the relative rotation ($R$) and translation ($t$) between two camera views is essential for tasks like Structure-from-Motion (SfM) and visual odometry. The standard approach involves solving for the essential matrix using at least five point correspondences. However, this process is sensitive to noise and requires a high number of correct matches. Polarimetric imaging offers a promising alternative by providing partial information about 3D surface normals at every pixel. While this data is subject to refractive index uncertainty and a 180-degree ($\pi$) orientation ambiguity, it provides powerful geometric cues that can simplify the pose estimation problem.
Methods
We propose a minimal solver that requires only two point correspondences by integrating polarimetric constraints.
The 2-Point Solver and Ambiguity Resolution
Polarization data provides the orientation of the surface normal but leaves the zenith angle dependent on the unknown refractive index ($n$) and the azimuth angle subject to a $\pi$-ambiguity. For two points across two views, there are $2^2 \times 2^2 = 16$ possible normal configurations. Our solver evaluates these configurations within a RANSAC framework to find the pose that maximizes consensus among all matches.
Joint Optimization
To refine the initial estimate, we perform a local optimization using the Levenberg-Marquardt algorithm. This stage simultaneously optimizes the camera pose and the refractive index ($n$) for each point. The objective function minimizes both the Sampson error (reprojection distance) and the alignment error between the estimated normals and those derived from polarization.
Key Results
The proposed method was evaluated against the classical 5-point algorithm using synthetic scenes and real-world datasets (VASE, BALLOON, TALLVASE).
On synthetic data, the 2-point solver demonstrated superior resilience to image noise. As shown in the figure above, while the error for the 5-point algorithm grows rapidly with noise, our optimized 2-point solver maintains significantly lower mean errors for both rotation and translation. Quantitatively, the 2-point solver achieved a rotation error of 1.80° compared to 4.95° for the 5-point baseline under standard noise levels. In real-world tests on the TALLVASE dataset, our method reduced rotation and translation errors by 44.39% and 24.94%, respectively.
Conclusion
By incorporating polarimetric surface normals, we have developed a 2-point relative pose solver that outperforms traditional 5-point methods in both efficiency and robustness. While currently limited to dielectric materials with diffuse reflections, this work paves the way for more robust geometric estimation in diverse lighting conditions. Future research will explore extensions to specular surfaces and metallic materials.
Summary updated