Shading Meets Motion: Self-supervised Indoor 3D Reconstruction Via Simultaneous Shape-from-Shading and Structure-from-Motion

Guoyu LuCVPR 2025 · DOI

Abstract

Shading-SfM-Net is a novel self-supervised framework that addresses the "textureless room" challenge in indoor 3D reconstruction. By integrating Shape-from-Shading (SfS) and Structure-from-Motion (SfM), the model leverages shading cues to reconstruct geometry where traditional feature-matching fails. It achieves state-of-the-art performance on NYUv2 and ScanNet datasets without requiring ground-truth depth labels.

The Challenge of Indoor Reconstruction

Traditional Structure-from-Motion (SfM) algorithms rely heavily on distinct textures and edges to match pixels across video frames. However, indoor environments often feature large, flat, white walls and smooth floors. In these "low-texture" regions, standard self-supervised models frequently fail, producing infinite or random depth estimates because the photometric error remains near zero regardless of the depth. Shading-SfM-Net addresses this by incorporating how light interacts with surface shape (shading) alongside traditional motion cues.

Methods: The Shading-SfM-Net Architecture

The proposed framework consists of two parallel pathways that inform one another to create a more robust reconstruction.

Overview of the Shading-SfM-Net architecture showing the SfM and SfS pathways and their joint optimization losses.
  1. The SfM Pathway: This pathway estimates depth maps and camera ego-motion from monocular video sequences. It utilizes BRISK keypoints to focus on reliable areas and an Adaptive Brightness module to handle inter-frame lighting variations.
  2. The SfS Pathway: This pathway performs "inverse rendering," decomposing a single image into its intrinsic components: Albedo (base color), Normals (surface orientation), and Lighting.

Joint Optimization

The innovation of Shading-SfM-Net lies in the joint optimization of these pathways through several novel constraints:

  • Surface Shape Consistency: Ensures surface normals from the SfS pathway match the depth map gradients from the SfM pathway.
  • 3D Geometric Registration Loss: Registers point clouds from both pathways against each other to ensure unified geometry.
  • Layout Consistency: Enforces that reconstructed scenes preserve dominant structural features like wall-floor boundaries.
  • Curved Surface Fitting: Uses superpixel segments from the albedo map to identify physical surfaces and fit general curved geometry, allowing for smoother depth transitions than simple planar assumptions.

Key Results

The model was evaluated on major indoor datasets including NYUv2 and ScanNet. Shading-SfM-Net achieves state-of-the-art (SOTA) results for unsupervised monocular depth estimation.

Qualitative depth estimation results on indoor scenes, demonstrating the model's ability to handle textureless regions compared to baselines.
  • NYUv2 Performance: The model achieved an Absolute Relative Error (AbsRel) of 0.129, significantly outperforming previous unsupervised leaders like RA-Depth (0.152) and MovingIndoor (0.208).
  • ScanNet Performance: It maintained a lead with an AbsRel of 0.151, proving effectiveness across diverse indoor distributions.
  • Qualitative Superiority: Visual results demonstrate that while other models produce "stretching" or "warping" artifacts on white walls, Shading-SfM-Net produces clean, geometrically accurate surfaces.
  • Generalization: A model trained only on NYUv2 performed remarkably well on ScanNet without any fine-tuning, suggesting the learned shading and motion cues are universal.

Conclusion and Future Work

Shading-SfM-Net demonstrates that combining shading and motion cues provides a "best of both worlds" solution for indoor reconstruction. While the current model assumes Lambertian (matte) surfaces, future work will focus on handling complex, non-uniform lighting and non-Lambertian materials like mirrors and polished floors to create a truly universal indoor 3D reconstruction system.

Summary updated

BibTeX
@inproceedings{lu2025shading,
  author = {Guoyu Lu},
  title = {{Shading Meets Motion: Self-supervised Indoor 3D Reconstruction Via Simultaneous Shape-from-Shading and Structure-from-Motion}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.01539},
}
Figure