Polarimetric Relative Pose Estimation

Zhaopeng Cui, Viktor Larsson, Marc Pollefeys

Abstract

Relative pose estimation is a cornerstone of 3D reconstruction and computer vision. Traditional methods, such as the 5-point algorithm, rely solely on 2D point correspondences. This paper introduces a novel polarimetric relative pose estimation framework that utilizes the surface normal information encoded in polarized light. By leveraging these additional geometric constraints, we derive a minimal 2-point solver that is more robust to noise and requires fewer correspondences than classical approaches. We further propose a joint optimization method to handle unknown material properties and orientation ambiguities. Our results on synthetic and real-world datasets demonstrate significant improvements in accuracy, particularly in challenging, noisy scenarios.

Introduction

Estimating the relative rotation ($R$) and translation ($t$) between two camera views is essential for tasks like Structure-from-Motion (SfM) and visual odometry. The standard approach involves solving for the essential matrix using at least five point correspondences. However, this process is sensitive to noise and requires a high number of correct matches. Polarimetric imaging offers a promising alternative by providing partial information about 3D surface normals at every pixel. While this data is subject to refractive index uncertainty and a 180-degree ($\pi$) orientation ambiguity, it provides powerful geometric cues that can simplify the pose estimation problem.

Methods

We propose a minimal solver that requires only two point correspondences by integrating polarimetric constraints.

Geometric setup for relative pose estimation showing cameras $c_1, c_2$ and surface normals derived from polarization.

The 2-Point Solver and Ambiguity Resolution

Polarization data provides the orientation of the surface normal but leaves the zenith angle dependent on the unknown refractive index ($n$) and the azimuth angle subject to a $\pi$-ambiguity. For two points across two views, there are $2^2 \times 2^2 = 16$ possible normal configurations. Our solver evaluates these configurations within a RANSAC framework to find the pose that maximizes consensus among all matches.

Joint Optimization

To refine the initial estimate, we perform a local optimization using the Levenberg-Marquardt algorithm. This stage simultaneously optimizes the camera pose and the refractive index ($n$) for each point. The objective function minimizes both the Sampson error (reprojection distance) and the alignment error between the estimated normals and those derived from polarization.

Key Results

The proposed method was evaluated against the classical 5-point algorithm using synthetic scenes and real-world datasets (VASE, BALLOON, TALLVASE).

Mean angular error for rotation (a) and translation (b) as a function of image noise ($\sigma_{xy}$), comparing 5-point and 2-point solvers.

On synthetic data, the 2-point solver demonstrated superior resilience to image noise. As shown in the figure above, while the error for the 5-point algorithm grows rapidly with noise, our optimized 2-point solver maintains significantly lower mean errors for both rotation and translation. Quantitatively, the 2-point solver achieved a rotation error of 1.80° compared to 4.95° for the 5-point baseline under standard noise levels. In real-world tests on the TALLVASE dataset, our method reduced rotation and translation errors by 44.39% and 24.94%, respectively.

Conclusion

By incorporating polarimetric surface normals, we have developed a 2-point relative pose solver that outperforms traditional 5-point methods in both efficiency and robustness. While currently limited to dielectric materials with diffuse reflections, this work paves the way for more robust geometric estimation in diverse lighting conditions. Future research will explore extensions to specular surfaces and metallic materials.

Summary updated

Abstract

In this paper we consider the problem of relative pose estimation from two images with per-pixel polarimetric information. Using these additional measurements we derive a simple minimal solver for the essential matrix which only requires two point correspondences. The polarization constraints allow us to pointwise recover the 3D surface normal up to a two-fold ambiguity for the diffuse reflection. Since this ambiguity exists per point, there is a combinatorial explosion of possibilities. However, since our solver only requires two point correspondences, we only need to consider 16 configurations when solving for the relative pose. Once the relative orientation is recovered, we show that it is trivial to resolve the ambiguity for the remaining points. For robustness, we also propose a joint optimization between the relative pose and the refractive index to handle the refractive distortion. In experiments, on both synthetic and real data, we demonstrate that by leveraging the additional information available from polarization cameras, we can improve over classical methods which only rely on the 2D-point locations to estimate the geometry. Finally, we demonstrate the practical applicability of our approach by integrating it into a state-of-the-art global Structure-from-Motion pipeline.
BibTeX
@inproceedings{cui2019polarimetric,
  author = {Zhaopeng Cui and Viktor Larsson and Marc Pollefeys},
  title = {{Polarimetric Relative Pose Estimation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00276},
}
Figure