Introduction
Recovering the 3D shape of an object—specifically its surface normals—using the polarization of reflected light is a powerful technique known as Shape from Polarization (SfP). Unlike standard imaging, polarization cameras can capture geometric details even on featureless surfaces. However, traditional SfP methods face two major hurdles: the $\pi$-ambiguity, where a single measurement can correspond to two different surface orientations, and the orthographic trap, where models incorrectly assume parallel light rays, leading to perspective-based errors.
This paper introduces a deep-learning framework that combines stereo polarization imaging to overcome these limitations, reconstructing high-accuracy surface normals and depth maps simultaneously.
Proposed Methodology
The proposed architecture integrates Convolutional Neural Networks (CNNs) for local feature extraction and Vision Transformers (ViTs) for global context. The pipeline follows a coarse-to-fine refinement strategy, processing left and right polarization images to output a high-quality normal map and a disparity (depth) map.
Two key innovations drive the system's performance:
- Shape Consistency-based Mask Prediction (SCMP): This module performs a "sanity check" by comparing surface normals derived from polarization (detailed but ambiguous) with those derived from stereo disparity (coarse but globally consistent). When a mismatch is detected, the module masks unreliable features, allowing the Transformer to fill in the area using global context.
- Viewing Direction-aided Positional Encoding (VDPE): To resolve the orthographic trap, the authors encode the unique viewing direction of the ray hitting each pixel. This allows the network to mathematically account for perspective distortion, significantly improving the accuracy of the underlying polarization physics.
Real-World Stereo SfP Dataset
Due to the lack of suitable existing data, the researchers established a new real-world stereo SfP dataset. It contains 2,450 pairs of polarization images covering over 50 different objects with diverse shapes and materials. Ground-truth normals and disparity maps were obtained using a high-accuracy stereo structured-light system under controlled, unpolarized illumination.
Key Results and Insights
The core insight of this work is that stereo geometry and polarization physics are complementary: stereo provides the "rough truth" that disambiguates the "fine details" of polarization. This synergy leads to state-of-the-art results:
- Accuracy: The method achieved a Mean Angular Error (MAE) of 13.22°, nearly halving the error of previous deep-learning SfP methods (which ranged from 22.27° to 24.71°).
- Precision: 46.2% of predicted pixels had an error of less than 11.25°, a significant improvement over the 28.8% achieved by the next best method.
- Robustness: By utilizing stereo polarization, the system is more resilient to lighting changes than monocular methods, as redundant information from two viewpoints maintains stable shape estimates.
Conclusion
This framework demonstrates that combining stereo vision with polarimetric physical priors can resolve long-standing ambiguities in 3D reconstruction. While currently tested under controlled unpolarized light, future work aims to extend this robustness to complex real-world environments and multi-view setups for industrial inspection and robotics.
Summary updated