Abstract
In this paper, a method of a novel combination photometric stereo which can estimate surface normals precisely even for images including shadows and specular reflection is proposed. Assuming that the number of input images for photometric stereo is more than three, the proposed method can exclude pixels affected by shadows and specular reflection by analyzing distributions of albedos and normal vectors computed from $nC_3$ combinations for $n$ input images. In these distributions, the proposed method defines a novel value “compactness”. The compactness indicates the degree of concentration of albedos and surface normals, which should be the same values if all pixel intensities of input images perfectly obey the Lambertian model without any error. Finally, pixels which are included in neither shadows nor specular reflection are chosen by voting using the compactness. The proposed method is experimentally verified to provide accurate surface normals in the presence of shadows and specular reflection, demonstrating superior accuracy compared to previous works. Moreover, a small device has been developed which supplies eight images varying in light positions and can be attached to smartphones, verifying the practical utility of the method.
Introduction
Photometric Stereo (PS) is a fundamental technique in computer vision used to reconstruct the 3D shape of an object by capturing multiple images under different lighting directions. While highly effective in controlled environments, standard PS relies on the Lambertian reflectance model, which assumes surfaces are perfectly matte. In real-world scenarios, this assumption is frequently violated by shadows and specular highlights, leading to significant reconstruction errors.
Existing robust PS methods often involve complex optimizations or require a large number of input images. This paper introduces a "Combination Photometric Stereo" approach that utilizes the statistical distribution of surface estimates to identify and discard outliers efficiently.
Methodology: The Compactness Metric
The core innovation of the proposed method is the use of a "compactness" metric to evaluate the reliability of image subsets.
1. Combination Approach
For a set of $n$ input images, the algorithm considers every possible combination of three images ($nC_3$). For each pixel, each triplet produces a candidate for the surface normal (represented by components $p, q$) and the albedo ($\rho$).
2. 3D Distribution and Compactness
In an ideal Lambertian case, all triplets would yield identical $(p, q, \rho)$ values. However, triplets containing pixels affected by shadows or specularities will produce divergent results. By mapping these estimates into a 3D space, the authors observed that "clean" data points form a dense cluster, while corrupted data points appear as scattered outliers.
The "compactness" is defined as the density of points within a specific threshold in this 3D space. By identifying the most compact region, the algorithm can pinpoint the most reliable lighting directions for each specific surface point.
3. Voting and Selection
A voting mechanism is employed to select the final set of pixels. Each triplet in the compact cluster "votes" for its constituent images. The images with the highest votes are then used in a final least-squares estimation to determine the precise surface normal and albedo.
Experimental Evaluation
The researchers validated their approach using both synthetic and real-world datasets.
Synthetic Results
Using a 3D "Hippo" model rendered with complex lighting, the proposed method achieved a Root Mean Squared Error (RMSE) for surface normals of 3.93. This is a significant improvement over conventional PS (RMSE 7.76) and other robust methods like those by Miyazaki et al. (RMSE 4.93).
Real-World Smartphone Application
To demonstrate practical utility, the authors developed a custom smartphone attachment featuring 8 LEDs and a magnifying lens. This portable setup was used to scan a 50-yen coin. The results showed that the compactness-based method could successfully recover fine inscriptions and surface textures that were otherwise obscured by the metallic coin's specular highlights in standard PS reconstructions.
Conclusion
The proposed Combination Photometric Stereo method offers a robust solution for 3D shape estimation in the presence of non-Lambertian effects. By incorporating albedo into a 3D clustering analysis and using a voting-based selection process, the method achieves high accuracy without the need for heavy computational overhead. The successful demonstration with a smartphone-based device highlights its potential for accessible, high-precision 3D scanning in mobile applications.
Summary updated