Combination Photometric Stereo Using Compactness of Albedo and Surface Normal

Naoto Ienaga, Hideo Saitô, Masayoshi Shimizu, Kouichi Teduka, Yasumasa IwamuraJournal of the Japan Society for Precision Engineering 2015 · DOI

Abstract

In this paper, a method of a novel combination photometric stereo which can estimate surface normals precisely even for images including shadows and specular reflection is proposed. Assuming that the number of input images for photometric stereo is more than three, the proposed method can exclude pixels affected by shadows and specular reflection by analyzing distributions of albedos and normal vectors computed from $nC_3$ combinations for $n$ input images. In these distributions, the proposed method defines a novel value “compactness”. The compactness indicates the degree of concentration of albedos and surface normals, which should be the same values if all pixel intensities of input images perfectly obey the Lambertian model without any error. Finally, pixels which are included in neither shadows nor specular reflection are chosen by voting using the compactness. The proposed method is experimentally verified to provide accurate surface normals in the presence of shadows and specular reflection, demonstrating superior accuracy compared to previous works. Moreover, a small device has been developed which supplies eight images varying in light positions and can be attached to smartphones, verifying the practical utility of the method.

Introduction

Photometric Stereo (PS) is a fundamental technique in computer vision used to reconstruct the 3D shape of an object by capturing multiple images under different lighting directions. While highly effective in controlled environments, standard PS relies on the Lambertian reflectance model, which assumes surfaces are perfectly matte. In real-world scenarios, this assumption is frequently violated by shadows and specular highlights, leading to significant reconstruction errors.

Existing robust PS methods often involve complex optimizations or require a large number of input images. This paper introduces a "Combination Photometric Stereo" approach that utilizes the statistical distribution of surface estimates to identify and discard outliers efficiently.

Methodology: The Compactness Metric

The core innovation of the proposed method is the use of a "compactness" metric to evaluate the reliability of image subsets.

Overview of the proposed Combination Photometric Stereo pipeline.

1. Combination Approach

For a set of $n$ input images, the algorithm considers every possible combination of three images ($nC_3$). For each pixel, each triplet produces a candidate for the surface normal (represented by components $p, q$) and the albedo ($\rho$).

2. 3D Distribution and Compactness

In an ideal Lambertian case, all triplets would yield identical $(p, q, \rho)$ values. However, triplets containing pixels affected by shadows or specularities will produce divergent results. By mapping these estimates into a 3D space, the authors observed that "clean" data points form a dense cluster, while corrupted data points appear as scattered outliers.

Clustering of albedo and surface normal estimates in 3D space. Blue circles represent the compact cluster of valid estimates, while red triangles are outliers.

The "compactness" is defined as the density of points within a specific threshold in this 3D space. By identifying the most compact region, the algorithm can pinpoint the most reliable lighting directions for each specific surface point.

3. Voting and Selection

A voting mechanism is employed to select the final set of pixels. Each triplet in the compact cluster "votes" for its constituent images. The images with the highest votes are then used in a final least-squares estimation to determine the precise surface normal and albedo.

Experimental Evaluation

The researchers validated their approach using both synthetic and real-world datasets.

Synthetic Results

Using a 3D "Hippo" model rendered with complex lighting, the proposed method achieved a Root Mean Squared Error (RMSE) for surface normals of 3.93. This is a significant improvement over conventional PS (RMSE 7.76) and other robust methods like those by Miyazaki et al. (RMSE 4.93).

Real-World Smartphone Application

To demonstrate practical utility, the authors developed a custom smartphone attachment featuring 8 LEDs and a magnifying lens. This portable setup was used to scan a 50-yen coin. The results showed that the compactness-based method could successfully recover fine inscriptions and surface textures that were otherwise obscured by the metallic coin's specular highlights in standard PS reconstructions.

Conclusion

The proposed Combination Photometric Stereo method offers a robust solution for 3D shape estimation in the presence of non-Lambertian effects. By incorporating albedo into a 3D clustering analysis and using a voting-based selection process, the method achieves high accuracy without the need for heavy computational overhead. The successful demonstration with a smartphone-based device highlights its potential for accessible, high-precision 3D scanning in mobile applications.

Summary updated

Abstract

In this paper, a method of a novel combination photometric stereo which can estimate surface normals precisely even for images including shadows and specular reflection is proposed. Assuming that the number of input images for photometric stereo is more than three, the proposed method can exclude pixels affected by shadows and specular reflection by analyzing distributions of albedos and normal vectors computed from nC3 combinations for n input images. In these distributions, the proposed method define a novel value “compactness”. The compactness indicates the degree of concentration of albedos and surface normals, which should be the same values if all pixel intensities of input images perfectly obey Lambertian model without any error. Finally pixels which are included in neither shadows nor specular reflection are chosen by voting using the compactness. The proposed method is experimentally verified that it can provide accurate surface normals in the presence of shadows and specular reflection and it is superior to with better accuracy than previous works. Moreover a small device have been developed which supplies eight images varying in light positions and can be attached to smartphones. A possibility of practical use of the proposed method with the device is also verified.
BibTeX
@article{ienaga2015combination,
  author = {Naoto Ienaga and Hideo Saitô and Masayoshi Shimizu and Kouichi TEDUKA and Yasumasa Iwamura},
  title = {{Combination Photometric Stereo Using Compactness of Albedo and Surface Normal}},
  journal = {Journal of the Japan Society for Precision Engineering},
  year = {2015},
  doi = {10.2493/jjspe.81.1154},
}
Figure