Abstract
Despite significant advancements in single-view intrinsic image decomposition, a domain disparity exists due to the limited information that can be obtained from a single-view image and the ill-posed nature of the problem of intrinsic image decomposition. Multi-view images offer an alternative method to circumvent the ambiguity present in 2D intrinsic image decomposition. Building on the concepts of multi-view intrinsic images and recent neural rendering techniques, we propose Intrinsic-GS, a multiview intrinsic image decomposition method utilizing Gaussian-splatting. To achieve this, we first augment each Gaussian ellipsoid with additional attributes (i.e., albedo, shading, and a residual term) to model the intrinsic radiance field. Next, we use several color-invariants and physics-based priors to jointly regularize the optimization of the intrinsic and composited radiance fields. Finally, we conduct experiments on both synthetic and real-world datasets, demonstrating stable intrinsic decomposition results across various (including non-Lambertian) objects and scenes.
Introduction
Intrinsic image decomposition (IID) is the process of separating an image into its fundamental components: albedo (the true color of the material) and shading (the effect of light and geometry). This is a critical task for AR/VR, scene editing, and object recognition. However, the problem is "ill-posed"—infinitely many combinations of light and color can result in the same pixel value.
While multi-view images help resolve some ambiguity, existing solutions face significant hurdles in speed and detail. NeRF-based methods can take over 8 hours to process a single object and often struggle to capture sharp texture details. Intrinsic-GS addresses these limitations by integrating intrinsic decomposition into the 3D Gaussian Splatting (3D-GS) framework, achieving state-of-the-art results in roughly 30 minutes.
Methodology
Intrinsic-GS builds on the high-speed rendering of 3D Gaussian Splatting by augmenting each 3D Gaussian ellipsoid with additional attributes beyond standard shape and opacity: albedo ($r$), shading ($s$), and a view-dependent residual term ($re$).
The pipeline utilizes three individual rasterizers to render albedo, shading, and residual images simultaneously. To regularize this optimization in a self-supervised manner, the model employs multi-level priors:
- Pixel-level: A reflectance sparsity prior based on chromaticity ensures pixels with similar colors have similar albedo.
- Edge-level: Cross-color ratios (CCR) preserve sharp boundaries between different materials.
- 3D-spatial level: K-means clustering on the 3D Gaussians' albedo attributes ensures material consistency across the entire 3D space.
- Shading Smoothness: A prior that encourages shading to change gradually, preventing texture details from "leaking" into the shading map.
Experiments and Results
The researchers evaluated Intrinsic-GS on major benchmarks including NeRF-blender, Shiny NeRF, and MIP-360.
Quantitative Performance
On the NeRF-blender dataset, Intrinsic-GS achieved a reflectance PSNR of 26.17 and an LPIPS of 0.0706, significantly outperforming Intrinsic-NeRF (24.40 / 0.3933) and GS-IR (25.99 / 0.4744). The method is approximately 20x faster than previous NeRF-based approaches.
Qualitative Superiority
As shown in the comparisons, the resulting albedo maps are noticeably sharper and free of "shading leakage" (where shadows are accidentally baked into the color map) compared to previous state-of-the-art methods. By explicitly modeling a "residual" term, the system effectively handles specular highlights and glossy reflections that usually break intrinsic decomposition.
Discussion and Conclusion
Intrinsic-GS demonstrates that moving from NeRF to Gaussian Splatting not only reduces training time from hours to minutes but also improves the robustness of intrinsic decomposition. Its "geometry-free" advantage makes it suitable for real-world scenes where perfect meshes are impossible to capture.
While the model currently assumes neutral incident light, it opens new doors for real-time creative workflows like 3D scene recoloring. Future work will focus on more robust color invariants to handle extremely complex, high-frequency reflective materials.
Summary updated