Geotensity Constraint for 3D Surface Reconstruction under Multiple Light Sources

Atsuto Maki, Charles WilesECCV 2000 · DOI

TL;DR

This paper extends the "Geotensity Constraint" for 3D surface reconstruction to handle scenes with multiple light sources, a significant limitation of previous work. By introducing a "surface interaction matrix," they enable robust segmentation of points based on their illumination subsets, allowing for accurate depth estimation even when lighting conditions change dynamically across the object's surface.

What Problem Does This Paper Address?

Reconstructing the 3D shape of an object from a series of 2D images is a fundamental problem in computer vision. A common assumption in many existing methods is that the object's surface has uniform intensity across different views, or that lighting conditions are constant. However, when an object moves or the camera changes its viewpoint, the illumination at any given point on the surface can change significantly, invalidating the constant intensity assumption. This makes it difficult to match corresponding points across images, which is crucial for 3D reconstruction. While the "Geotensity Constraint" was developed to address this by relating image intensities to geometry and lighting, it was primarily limited to scenarios with a single light source. This paper tackles the challenge of extending this powerful constraint to the more realistic scenario of multiple, potentially unknown, light sources.

Methods

The core of this paper is the extension of the Geotensity Constraint to handle multiple light sources. The authors propose a two-stage approach:

  1. Segmentation of Illumination Subspaces:
    • They introduce the surface interaction matrix (H), derived from the illumination matrix (I) using Singular Value Decomposition (SVD). This matrix captures the interaction between object surface properties and light source directions.
  2. Crucially, they show that block-diagonalizing H allows for the segmentation of sample points into clusters, where each cluster corresponds to a unique subset of light sources illuminating the object. This is a key theoretical contribution, enabling the identification of different illumination subspaces.
  3. A practical method for achieving this block diagonalization, even with noisy data, is proposed, involving iterative minimization and RANSAC for outlier rejection.
  4. Surface Reconstruction with Multiple Light Sources:
    • Once the illumination subspaces are identified, the authors estimate the light source directions for each subset independently.
  5. For each point on the object's surface, they then consider all possible combinations of these estimated light source directions to reconstruct the depth. This accounts for the fact that different points might be illuminated by different subsets of lights.
  6. The depth is estimated by minimizing an error function based on the Geotensity constraint, considering all valid light source combinations for that point.

Datasets

The paper uses synthetic data (a sphere) and real-world data (a statue of Julius Caesar) for evaluation. No new benchmarks are introduced.

Key Insights and Ideas

  • The "Surface Interaction Matrix" (H) is the key: This matrix elegantly encodes how different points on the object's surface are illuminated by different combinations of light sources, enabling their segmentation.
  • Block diagonalization of H is a powerful segmentation tool: This mathematical property directly translates to identifying distinct illumination subspaces on the object's surface.
  • Handling dynamic illumination is crucial for realistic 3D reconstruction: The paper demonstrates that by explicitly accounting for varying illumination subsets, more accurate and robust reconstructions can be achieved compared to methods that assume constant lighting.
  • Geotensity Constraint's generality: The paper shows that the underlying principle of the Geotensity Constraint can be extended beyond single light sources, making it a more versatile tool.

Key Results

  • The proposed method successfully reconstructs the 3D surface of a synthetic sphere under two light sources, demonstrating accurate depth estimation even in areas illuminated by different combinations of lights.
  • Experiments on a real statue of Julius Caesar show that the Geotensity constraint-based reconstruction is more robust to noise and textureless regions compared to traditional cross-correlation methods, particularly in areas with complex illumination changes.
  • The method effectively handles situations where different parts of the object are illuminated by different subsets of light sources, a common scenario in real-world applications.

Limitations and Future Work

  • Assumptions: The initial formulation assumes Lambertian surface properties and convex shapes (no self-shadowing). While the authors mention these can be relaxed, the presented results are based on these assumptions.
  • Number of light sources: The algorithm requires prior knowledge of the number of distinct illumination subspaces (m) on the object's surface.
  • Computational complexity: While the method is automatic, the search for depth and light source combinations can be computationally intensive, especially with a large number of light sources or frames.
  • Future Work: The authors suggest extending the algorithm to handle more general lighting conditions, including shadowing and inter-reflection. They also propose developing more efficient mechanisms for clustering sample points and searching for light source combinations, especially in the presence of noise and a large number of light sources. Statistical approaches and genetic algorithms are mentioned as potential avenues.

Summary updated

BibTeX
@inproceedings{maki2000geotensity,
  author = {Atsuto Maki and Charles Wiles},
  title = {{Geotensity Constraint for 3D Surface Reconstruction under Multiple Light Sources}},
  booktitle = {European Conference on Computer Vision},
  year = {2000},
  doi = {10.1007/3-540-45054-8\_47},
}
Figure