Shape and Illumination from Shading using the Generic Viewpoint Assumption

Daniel Zoran, Dilip Krishnan, José Bento, Bill FreemanNeurIPS 2014 · OpenReview

Introduction

Recovering the 3D shape of an object from a single 2D shading image is a classic "ill-posed" problem in computer vision. Because many different combinations of surface orientation and lighting can produce the exact same pixel values, researchers typically rely on heavy, hand-tuned priors—such as assuming surfaces must be very smooth or that light always comes from above.

This paper introduces a practical algorithm for "Shape from Shading" that avoids these traditional priors by leveraging the Generic Viewpoint Assumption (GVA). The GVA posits that a scene's appearance should be stable under small changes in perspective or lighting. By combining a novel linearized Spherical Harmonics model with a multi-scale optimization framework, the authors successfully recover 3D shape and illumination from a single image without relying on traditional geometry or light priors.

The Generic Viewpoint Assumption

The authors tackle the ambiguity of shape from shading using the GVA. The intuition is simple: if a specific 3D interpretation of an image only "works" from one exact, precarious angle, it is likely wrong.

Illustration of special vs. generic viewpoints

A "generic" and therefore more probable interpretation is one that remains stable even if you nudge the camera or the light source slightly. While this concept has existed in psychophysics for decades, turning it into a functional computer vision algorithm has historically been computationally prohibitive.

Methodology

The proposed framework avoids the need for explicit priors on shape or illumination by maximizing "genericity." The approach consists of three main components:

  1. Linearized Spherical Harmonics (SH): Standard SH models for lighting are non-linear and non-convex, making them difficult to optimize. The authors linearize the log-shading model around local surface slope estimates. This simplification is the key mathematical step that makes the GVA tractable.
  2. The GVA Term: They derive a "genericity" term based on the Fisher Information Matrix. This term mathematically penalizes interpretations where the rendered image would change drastically under infinitesimal rotations of the object.
  3. ADMM Optimization: To solve for both depth ($Z$) and illumination ($L$) simultaneously, they use the Alternating Direction Method of Multipliers (ADMM). This breaks the complex global problem into smaller, manageable sub-problems: one for the image data fit, one for GVA stability, and one to ensure the resulting surface is physically "integrable" (meaning the slopes actually form a valid 3D shape).

Key Results and Insights

The model was evaluated on datasets including the MIT/Berkeley Intrinsic Images (18 models, natural light) and the Harvard Dataset (7 models, lab lighting), comparing results against the state-of-the-art "Shape, Illumination, and Reflectance from Shading" (SIFS) algorithm.

Quantitative comparison of GVA against SIFS
  • Stability as a Regularizer: The most profound insight is that stability is a powerful substitute for smoothness. Instead of forcing the algorithm to make the surface smooth, the GVA term forces the surface to be robust to small rotations.
  • Competitive Performance: The GVA-based method achieved results comparable to SIFS, despite SIFS using significantly more information (like contour normals and explicit shape priors).
  • Superior Light Estimation: The GVA term proved particularly effective for recovering illumination, often outperforming SIFS in light-source accuracy (L-MSE).
  • Robustness: On the Harvard dataset, the GVA method outperformed SIFS, suggesting it generalizes better to models it wasn't specifically tuned for. Visually, the method recovers dominant structures that purely smoothness-based priors tend to "wash out" or over-smooth.

Limitations and Future Work

While a significant step forward, the approach has limitations. The current implementation in unoptimized MATLAB takes about 7 minutes per image—much faster than the exhaustive search previously theorized for GVA, but not real-time. Additionally, the model assumes the object has a uniform color (constant albedo) and cannot yet handle varying materials or painted patterns. Future work may investigate how GVA can coexist with and complement traditional priors like sparsity or smoothness to achieve even higher accuracy.

Summary updated

Abstract

The Generic Viewpoint Assumption (GVA) states that the position of the viewer or the light in a scene is not special. Thus, any estimated parameters from an observation should be stable under small perturbations such as object, viewpoint or light positions. The GVA has been analyzed and quantified in previous works, but has not been put to practical use in actual vision tasks. In this paper, we show how to utilize the GVA to estimate shape and illumination from a single shading image, without the use of other priors. We propose a novel linearized Spherical Harmonics (SH) shading model which enables us to obtain a computationally efficient form of the GVA term. Together with a data term, we build a model whose unknowns are shape and SH illumination. The model parameters are estimated using the Alternating Direction Method of Multipliers embedded in a multi-scale estimation framework. In this prior-free framework, we obtain competitive shape and illumination estimation results under a variety of models and lighting conditions, requiring fewer assumptions than competing methods.
BibTeX
@inproceedings{zoran2014shape,
  author = {Daniel Zoran and Dilip Krishnan and José Bento and Bill Freeman},
  title = {{Shape and Illumination from Shading using the Generic Viewpoint Assumption}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2014},
  url = {https://openreview.net/forum?id=rk-Xp8bOWB},
}
Figure