Abstract
Multispectral Photometric Stereo (MPS) aims to recover 3D surface geometry from a single multispectral image. However, the technique is often hampered by the "reflectance-shape ambiguity," where it is difficult to distinguish between variations in surface orientation and variations in material color. While traditional methods rely on spatial smoothness constraints to resolve this, such assumptions fail for objects with complex textures. This paper introduces a per-pixel solution that utilizes strategic lighting configurations and a Local Linear Approximation (LLA) of spectral reflectance. By canceling out unknown reflectance terms through clever geometric design, the method achieves high-accuracy shape and reflectance recovery without spatial priors.
Introduction
Photometric Stereo (PS) is a cornerstone of computer vision, enabling high-resolution topography estimation by observing a surface under varying illumination. Multispectral Photometric Stereo (MPS) extends this by using different wavelengths of light simultaneously, potentially enabling "one-shot" acquisition. The primary challenge in MPS is the "Catch-22": a dark pixel could represent a surface tilted away from the light or a material with low reflectance at that wavelength. Most existing solutions impose spatial smoothness, assuming neighboring pixels share similar properties. This paper proposes a robust alternative that treats every pixel independently.
Methodology
The proposed framework relies on two key pillars: geometric lighting design and spectral smoothness.
- Strategic Lighting Configuration: Light sources are arranged in specific subsets where their direction vectors are linearly dependent. For instance, in a five-light subset, three "anchor" lights are supplemented by two intermediate lights placed at their midpoints.
- Local Linear Approximation (LLA): Real-world materials typically exhibit smooth spectral reflectance. The authors assume that over small wavelength intervals (10–20 nm), reflectance can be modeled as a linear function.
By combining these elements, the authors derive a mathematical formulation where the unknown reflectance terms cancel out, allowing the surface normal to be solved directly from image intensities. To handle noise and specularities, a simultaneous optimization is performed across the spectral domain, treating specular highlights as outliers via a moving median filter.
Validating the LLA Assumption
The core of the method—the Local Linear Approximation—was validated using the Munsell Color Matt Dataset, comprising 1,269 material samples. As shown in the experimental analysis, the regression error remains remarkably low for narrow observation intervals, confirming that the linear model is a physically sound approximation for natural materials.
Results and Discussion
The method was evaluated on both synthetic and real-world datasets. In synthetic tests (e.g., "Bunny" and "Dragon" models with high-frequency textures), the per-pixel approach achieved a Mean Angular Error (MAE) for surface normals as low as 0.085, significantly outperforming methods that rely on spatial masks. Robust simultaneous optimization also improved the Root Mean Square Error (RMSE) of recovered spectral reflectance from ~0.5 to ~0.3. Real-world experiments using a custom hardware setup confirmed the method's robustness to complex color patterns and specularities, even when the spectral sampling interval was increased to 20nm.
Conclusion
This research demonstrates that the reflectance-shape ambiguity in MPS can be resolved at the pixel level by leveraging the interplay between lighting geometry and spectral properties. By moving away from spatial smoothness constraints, the method enables accurate 3D reconstruction of highly textured and multi-colored objects, paving the way for more robust one-shot photometric stereo applications.
Summary updated