Optimal illumination directions for faces and rough surfaces for single and multiple light imaging using class-specific prior knowledge

Vasileios Argyriou, Stefanos Zafeiriou, Maria PetrouCVIU 2014 · DOI

TL;DR

This paper introduces two novel methods for determining optimal illumination directions for 3D surface reconstruction using photometric stereo. By leveraging class-specific prior knowledge about surface normal distributions (e.g., for faces), these methods significantly improve reconstruction accuracy compared to standard, uniform lighting setups. This is crucial for applications requiring detailed and reliable 3D shape recovery.

What Problem Does This Paper Address?

Photometric stereo is a powerful technique for recovering 3D surface shape and reflectance by analyzing how an object's appearance changes under different lighting conditions. However, the quality of the reconstructed surface is highly sensitive to the chosen illumination directions. Suboptimal lighting can lead to significant errors, particularly in areas with shadows or highlights, and can obscure fine surface details. While previous work has explored optimal lighting, it often assumed uniform surface normal distributions or lacked the ability to adapt to specific object classes. This paper addresses this gap by proposing methods that optimize illumination directions based on prior knowledge of the statistical distribution of surface normals for a given class of objects, such as human faces. This allows for more tailored and effective lighting strategies.

Methods

The authors propose two main methodologies:

  • Method 1: Pixel Domain Optimization: This approach optimizes illumination directions by maximizing an objective function that balances three criteria: enhancing surface detail visibility, minimizing shadows, and minimizing highlights (for the Phong model). It operates directly in the pixel domain and uses simulated annealing to find the optimal light source positions. This method is applicable to both Lambertian and Phong reflectance models.
  • Method 2: Gradient Space Optimization using Isophotes: This method leverages the fact that denser isophotes (contours of constant image intensity) in the gradient space lead to more accurate surface normal estimations. By knowing the distribution of surface normals for a class, it aims to find illumination directions that create dense isophotes in regions corresponding to the most common surface orientations. This approach is particularly useful for Lambertian surfaces and relies on maximizing the gradient magnitudes of reflectance maps.

Both methods incorporate "class-specific prior knowledge" by using statistical distributions of surface normals derived from representative models of the object class (e.g., a 3D face model). For multiple light sources, an alternating optimization strategy is employed.

Datasets

The paper utilizes a combination of simulated and real-world data:

  • Simulated Surfaces: Isotropic and anisotropic surfaces with varying textures were generated to test the methods. These surfaces had known ground-truth normal distributions and albedos.
  • Real Faces: 3D models of human faces were used, both with and without ground-truth data.
  • Real Objects: Bottles and potatoes were used for evaluation, with ground-truth data obtained from a 3dMD imaging system.

No new datasets were introduced; instead, existing benchmarks and data types were used to validate the proposed approaches.

Key Insights and Ideas

  • Class-Specific Lighting is Key: The most significant insight is that "one-size-fits-all" lighting configurations are suboptimal. Tailoring illumination to the expected surface normal distribution of an object class (like faces) dramatically improves reconstruction quality.
  • Balancing Competing Objectives: Method 1 elegantly balances the need to reveal detail with the need to suppress unwanted artifacts like shadows and highlights, demonstrating that these objectives can be optimized simultaneously.
  • Isophote Density Matters: Method 2 highlights the crucial role of isophote density in the gradient space for accurate normal recovery, providing a theoretical basis for optimizing illumination based on this principle.
  • Prior Knowledge is Powerful: The success of both methods hinges on the availability of prior knowledge about surface normal distributions, which can be obtained from generic models or representative samples of an object class.

Key Results

The proposed methods consistently outperformed default, uniformly distributed illumination settings across various experiments:

  • Reduced Angular Error: For simulated anisotropic surfaces and real faces, the proposed methods achieved significantly lower mean angular errors (MAE) in surface normal estimation, often reducing errors from several degrees to near zero. For example, for faces, MAE dropped from over 5 degrees to around 0.1-0.3 degrees.
  • Improved Albedo Accuracy: Similarly, the mean absolute difference (MAD) in albedo estimation was substantially reduced, indicating more accurate reflectance recovery.
  • Enhanced Detail and Reduced Artifacts: Qualitative results showed that reconstructions using optimal lighting exhibited fewer shadows, more revealed details (especially in challenging areas like under the jaw), and less pronounced seams between illuminated and shadowed regions.
  • Robustness: The methods demonstrated effectiveness for both Lambertian and Phong models, and across different surface types (isotropic, anisotropic, faces, objects).

Limitations and Future Work

  • Prior Knowledge Requirement: The primary limitation is the reliance on prior knowledge of surface normal distributions. While the paper suggests obtaining this from generic models or samples, acquiring this information for entirely novel object classes might be challenging.
  • Self-Shadows vs. Cast Shadows: Method 1 primarily addresses self-shadows, with cast shadows not being explicitly modeled in the simulations.
  • Computational Cost: While the optimization is performed offline, it can be computationally intensive.
  • Future Work: The authors suggest extending the methods to handle more complex reflectance models and exploring ways to automatically infer class-specific normal distributions for unknown objects. Further investigation into the impact of different parameter choices (λ, κ) on performance is also implied.

Summary updated

BibTeX
@article{argyriou2014optimal,
  author = {Vasileios Argyriou and Stefanos Zafeiriou and Maria Petrou},
  title = {{Optimal illumination directions for faces and rough surfaces for single and multiple light imaging using class-specific prior knowledge}},
  journal = {Comput. Vis. Image Underst.},
  year = {2014},
  doi = {10.1016/j.cviu.2014.01.012},
}
Figure