Abstract
In this paper, we present a technique for estimating the geometry and reflectance of objects using only a camera, flashlight, and optionally a tripod. We propose a simple data capture technique in which the user goes around the object, illuminating it with a flashlight and capturing only a few images. Our main technical contribution is the introduction of a recursive neural architecture, which can predict geometry and reflectance at high resolution (1024x1024) given an input image and estimated geometry from previous steps. This recursive architecture, termed RecNet, is trained with 256x256 resolution but can easily operate on 1024x1024 images during inference. We show that our method produces more accurate surface normal and albedo, especially in regions of specular highlights and cast shadows, compared to previous approaches, given three or fewer input images.
Introduction
Capturing high-quality 3D geometry and material properties (reflectance) typically requires specialized equipment. Photometric Stereo (PS) systems produce excellent results but often necessitate expensive "light stages" in controlled dark rooms. Conversely, single-image Shape-from-Shading (SfS) methods often lack detail and fail to generalize to complex real-world objects.
This research bridges the gap by enabling high-resolution reconstruction using everyday items: a smartphone and a handheld flashlight. The proposed system generates detailed 1024×1024 surface normals, albedo, and roughness maps from as few as one to six images, significantly outperforming existing methods in consumer-grade capture scenarios.
Methods: The RecNet Architecture
The core innovation is RecNet, a recursive multi-resolution neural network designed to overcome the resolution limitations of standard deep learning models. Most existing solutions are restricted to low resolutions (e.g., 256×256) because their receptive fields do not scale well, leading to artifacts when applied to high-resolution images.
RecNet employs a recursive upsampling strategy:
- Initialization: A small network (InitNet) provides an initial estimate of shape and material at 32×32 resolution.
- Recursion: RecNet iteratively takes the previous estimate and a downsampled version of the original image to predict the next level of detail (64×64, 128×128, etc.) until reaching 1024×1024.
- Scale Invariance: By reusing the same weights at every step, the network's effective receptive field grows proportionally with the resolution, maintaining global context.
- Material Modeling: The system uses the Cook-Torrance model to predict albedo and roughness, relying on photometric cues (light and shadow) rather than simple visual priors.
Results and Evaluation
The system was evaluated using synthetic data, the standard DiLiGenT benchmark, and a new real-world dataset of 111 household objects. On the DiLiGenT benchmark, using only three images, the method achieved a Mean Angular Error (MAE) of 12.5°, nearly doubling the accuracy of the previous state-of-the-art, SDPS-Net (23.3°).
Qualitatively, RecNet produces clean 1024×1024 maps without the checkerboard artifacts common in standard architectures. It excels at separating material properties, effectively removing specular highlights and cast shadows from albedo maps to create realistic 3D renders.
Conclusion
While the system requires a relatively dim environment to ensure the flashlight is the dominant light source, it represents a major step toward accessible 3D capture. Future work may focus on improving roughness estimation for diverse materials and integrating normal estimation directly into mesh or depth prediction within the recursive loop.
Summary updated