Abstract
In this paper, we present a technique for estimating the geometry and reflectance of objects using only a camera, flashlight, and optionally a tripod. We propose a simple data capture technique in which the user goes around the object, illuminating it with a flashlight and capturing only a few images. Our main technical contribution is the introduction of a recursive neural architecture, which can predict geometry and reflectance at 2^{k}2^{k} resolution given an input image at 2^{k}2^{k} and estimated geometry and reflectance from the previous step at 2^{k-1}*2^{k-1}. This recursive architecture, termed RecNet, is trained with 256x256 resolution but can easily operate on 1024x1024 images during inference. We show that our method produces more accurate surface normal and albedo, especially in regions of specular highlights and cast shadows, compared to previous approaches, given three or fewer input images.
Introduction
The creation of high-quality 3D assets is essential for modern applications in e-commerce, AR/VR, and digital art. However, traditional methods for capturing shape and material properties often require specialized hardware, such as calibrated light stages, or are limited to low-resolution outputs. Photometric Stereo (PS) provides high-fidelity results but is cumbersome for home users, while single-image Shape-from-Shading (SfS) methods often lack the necessary detail and suffer from visual ambiguities. A primary challenge in deep learning-based 3D capture is scaling to high resolutions (e.g., 1024x1024) without losing global context or exceeding GPU memory limits.
Methodology
We address these challenges through a combination of a practical capture protocol and a novel recursive neural network architecture.
Weakly Calibrated Capture
Our approach utilizes a "weakly calibrated" setup where a user captures 1 to 6 images of an object using a standard camera and a handheld flashlight. Instead of precise calibration, the user only needs to provide rough lighting directions (e.g., "front-right"). We found that knowing the light direction within a ~12.5-degree margin of error provides sufficient constraints to significantly improve reconstruction quality over completely uncalibrated methods.
RecNet Architecture
To handle high-resolution inference, we introduce RecNet, a recursive multi-resolution architecture. Rather than predicting high-resolution maps in a single pass, the network iteratively refines its predictions. An initial InitNet produces a 32x32 map from a downsampled image. Subsequently, the RecNet block takes the next resolution's image and the upsampled previous prediction to generate a higher-resolution output. This process is repeated until the target resolution (up to 1024x1024) is reached. This design ensures that the network maintains a scale-agnostic receptive field, allowing global context to guide local detail refinement.
Results and Discussion
The effectiveness of our method was evaluated on the DiLiGenT benchmark and a custom dataset of 111 household objects.
Quantitative Performance
On the DiLiGenT dataset, our method achieved a Mean Angular Error (MAE) of 12.5° for surface normals using only three images, outperforming existing state-of-the-art methods like SDPS-Net (23.3°). Even in the single-image case, our model achieved an MAE of 16.7°, a significant improvement over previous baselines.
Qualitative Analysis
Our model excels at disentangling material properties, effectively separating cast shadows and specular highlights from the base albedo. By training on synthetic data where albedo and roughness were intentionally decoupled, the network learned to rely on photometric cues rather than memorized material correlations.
Conclusion
We have presented a practical system for high-resolution shape and material capture at home. By leveraging a recursive architecture and a weakly calibrated capture setup, we bridge the gap between professional light stages and consumer-grade single-image methods. While challenges remain in fully automating segmentation and improving roughness estimation, our approach provides a robust foundation for accessible 3D asset creation.
Summary updated