Perspective Shape from Shading with Non-Lambertian Reflectance

Oliver Vogel, Michael Breuß, Joachim WeickertDAGM-Symposium 2008 · DOI

TL;DR

This paper introduces a novel perspective shape-from-shading (SFS) model that accounts for non-Lambertian surfaces, a significant limitation of previous methods. By incorporating the Phong reflection model into a perspective SFS framework and developing an efficient numerical solver, they achieve more accurate and realistic 3D shape reconstructions from single images, even in the presence of noise.

What Problem Does This Paper Address?

The classic shape-from-shading (SFS) problem aims to recover the 3D shape of an object from a single 2D image. Traditional SFS methods often assume orthographic projection and Lambertian reflectance (i.e., surfaces that reflect light equally in all directions). While perspective projection models offer more realistic camera behavior and improved accuracy, extending them to handle non-Lambertian surfaces—which are far more common in the real world—has been a persistent challenge. Non-Lambertian surfaces exhibit complex reflectance properties, including specular highlights, which can lead to significant errors and ambiguities in shape reconstruction if not properly modeled. This paper addresses this gap by developing a perspective SFS model that explicitly accounts for these non-Lambertian effects, specifically using the Phong reflection model.

Methods

The core of the paper lies in extending the perspective SFS model to incorporate non-Lambertian reflectance. The authors start with a standard perspective SFS model for Lambertian surfaces, which is formulated as a Hamilton-Jacobi equation. They then integrate the Phong reflection model, which accounts for ambient, diffuse, and specular light components, into this framework. This leads to a new, more complex Hamilton-Jacobi equation.

To solve this equation efficiently, they employ a method of artificial time, introducing a pseudo-time variable and iterating until a steady state is reached. The spatial derivatives are discretized using a stable upwind-type scheme. For the nonlinear update step, they utilize a one-dimensional Newton method, which is known for its fast convergence. The algorithm is initialized with a simple guess derived from the image intensity and then iteratively refined.

Datasets

The paper evaluates its method on both synthetic and real-world images.

  • Synthetic Data: A "vase" dataset is used, for which the ground truth surface is known. This allows for quantitative evaluation of reconstruction accuracy. The vase is rendered with both Lambertian and Phong reflectance properties to demonstrate the model's capabilities.
  • Real-World Data: A photograph of a plastic cup is used. This experiment highlights the model's performance on real images where several assumptions of the model (e.g., uniform surface, single light source at the optical center) are violated.

No new datasets are introduced; existing benchmarks are used for evaluation.

Key Insights and Ideas

  • Phong Model for SFS: The most significant insight is the successful integration of the Phong reflection model into a perspective SFS framework. This moves SFS beyond the unrealistic Lambertian assumption, enabling more accurate shape recovery for a wider range of real-world surfaces.
  • Handling Specularities: The Phong model explicitly accounts for specular highlights, which are a major source of error in Lambertian SFS. The paper shows that by modeling these, the reconstruction quality improves dramatically, especially at these highlight regions.
  • Robustness to Noise: The proposed method demonstrates remarkable stability and good reconstruction quality even when the input images are significantly corrupted by Gaussian noise. This is crucial for practical applications where image acquisition is rarely perfect.
  • Efficient Numerical Scheme: The development of a stable and efficient numerical solver, leveraging artificial time and Newton's method, is key to making the complex non-Lambertian model computationally tractable.

Key Results

  • Quantitative Improvement: For the synthetic "vase" experiment, the Phong model achieved significantly lower average L₁ errors in depth ($u$) and its logarithm ($v$) compared to the Lambertian model. For instance, the relative error in $u$ was 4.62% with the Phong model versus 9.66% with the Lambertian model.
  • Qualitative Improvement: Reconstructions of the vase show that the Phong model preserves sharp transitions and details better, avoiding the smoothing artifacts common in Lambertian reconstructions. For the cup experiment, the Phong model produced a more accurate shape, particularly in recovering the flat background and avoiding artifacts at specular highlights, compared to the Lambertian model.
  • Noise Performance: Even with significant Gaussian noise (σ=5 and σ=10), the Phong model maintained good reconstruction quality, with errors only slightly increasing compared to the noiseless case.
  • Computational Efficiency: The computational times are reported to be "modest," only slightly slower than the Lambertian algorithm, which is impressive given the increased complexity of the model.

Limitations and Future Work

  • Model Assumptions: The authors acknowledge that the model still relies on several assumptions, such as a single light source at the optical center, and uniform surface properties. Real-world scenes often violate these assumptions (e.g., ambient lighting, multiple light sources, complex scene compositions). The cup experiment specifically highlights these violations.
  • Phong Model as Approximation: While better than Lambertian, the Phong model itself is an approximation of physical reality.
  • Future Work: The authors suggest extending the model to handle more complicated illumination models and improving algorithmic speedups. They also propose using the current model for automated document analysis.

Summary updated

BibTeX
@inproceedings{vogel2008perspective,
  author = {Oliver Vogel and Michael Breuß and Joachim Weickert},
  title = {{Perspective Shape from Shading with Non-Lambertian Reflectance}},
  booktitle = {DAGM-Symposium},
  year = {2008},
  doi = {10.1007/978-3-540-69321-5\_52},
}
Figure