TL;DR
This paper introduces "symmetric-light photometric stereo," a novel approach that uses a specific, symmetrical arrangement of four directional lights to accurately estimate surface normals without needing to know the exact light source directions or their distances. This method simplifies the setup and calibration process while achieving accuracy comparable to traditional, fully calibrated photometric stereo techniques.
What Problem Does This Paper Address?
Photometric stereo is a powerful technique for reconstructing the 3D shape of objects by analyzing how their appearance changes under different lighting conditions. Traditionally, this requires precise knowledge of the direction and intensity of each light source, which often involves a tedious and complex calibration step. While uncalibrated photometric stereo methods exist, they often suffer from ambiguities in the estimated surface normals. More recent approaches using ring-light configurations have helped constrain the problem, but they still may require assumptions about light source properties (like radius) or scene properties (like surface integrability) and can still exhibit ambiguities. This paper aims to bridge this gap by providing a method that is both simpler to set up and calibrate than traditional methods, yet robust enough to avoid the ambiguities inherent in many uncalibrated approaches.
Methods
The core innovation lies in the "symmetric-light configuration." Instead of arbitrary light placements, the authors propose using four point light sources arranged symmetrically with respect to the camera's optical axes. These lights are assumed to be infinitely far away, creating directional illumination. The key insight is that this symmetry generates independent constraints on the surface normal's components (x, y, and z).
The method then leverages a pair of pixels that share the same surface albedo but have different surface normals. By analyzing the differences in measurements from these pixels under the symmetric lighting, the authors derive a system of quadratic equations. Crucially, by assuming the lights lie on a circle (i.e., $l_x = l_y$), they can form a linear system that, when combined with the unit norm constraint for surface normals, leads to a solvable system. They employ a non-negative least squares (NNLS) approach to find the surface normals.
To make the method practical, they introduce a two-stage pixel-pair selection strategy:
- Albedo Clustering: They use chromaticity-based clustering to group pixels with similar albedos, as true albedo is unknown.
- Surface Normal Clustering: They cluster surface normals based on their elevation angles (derived from the measurements) to find pairs with distinct surface orientations.
Datasets
- Synthetic Data: Generated using Blender for scenes named PILLOW, TEDDY, and PLANT. These scenes were rendered with and without global illumination effects to test robustness.
- Real-World Data:
- DiLiGenT Dataset: A benchmark dataset for calibrated photometric stereo. The authors selected four light directions from the dataset that approximated the symmetric-light assumption.
- Custom Captured Data: Captured using their own setup with a CCD camera and LED arrays. Objects with simple (SPHERE, KITTEN) and complex (LIZARD) shapes, as well as spatially varying albedos (SHEEP, CHALK), were used.
Key Insights and Ideas
- Symmetry as a Constraint: The paper demonstrates that a specific symmetrical arrangement of lights, even without knowing their exact positions, provides sufficient constraints to resolve surface normal ambiguities. This is a significant departure from methods that rely on precise calibration or strong scene assumptions.
- Pixel Pairs Unlock Uniqueness: The clever use of pixel pairs with identical albedo but different surface normals is crucial. This allows the derived equations to isolate the surface normal information, leading to a unique solution.
- Simplification of Calibration: By eliminating the need for precise light source calibration, the method significantly reduces the complexity and effort required for photometric stereo applications.
- Robustness to Slight Deviations: Experiments show that the method remains reasonably accurate even when the light directions deviate slightly from the perfect symmetric configuration, suggesting practical applicability.
Key Results
- The proposed symmetric-light photometric stereo method achieves accuracy comparable to fully calibrated photometric stereo methods on both synthetic and real-world datasets.
- On the DiLiGenT dataset, the proposed method showed comparable accuracy to calibrated methods for scenes like BALL, BEAR, and POT2, even when the symmetric-light assumption was not perfectly met.
- For their custom dataset, the method also produced comparable results to calibrated methods, with some exceptions in regions with complex albedo variations or shadows that affected albedo clustering.
- The method is shown to be robust to noise in light directions, maintaining accuracy comparable to calibrated methods.
Limitations and Future Work
- Distant Light Assumption: The current method assumes lights are infinitely far away (directional lights). The authors acknowledge this as a limitation and propose extending the method to handle "near light" settings where lights are close to the scene.
- Albedo Approximation: The method relies on albedo clustering using chromaticity, which is an approximation. The paper suggests that improving albedo approximation and clustering could further enhance accuracy.
- Surface Normal Clustering: While effective, the accuracy of surface normal estimation is shown to be dependent on the quality of surface normal clustering, especially for scenes with limited distinct surface normals.
- Color vs. Grayscale: The current implementation converts color measurements to grayscale, potentially losing information. Future work could explore extending the method to handle color photometric stereo.
Summary updated