SPLiT: Single Portrait Lighting Estimation via a Tetrad of Face Intrinsics
Lists
Only you see this
Abstract
This paper proposes a novel pipeline to estimate a non-parametric environment map with high dynamic range from a single human face image. Lighting-independent and -dependent intrinsic images of the face are first estimated separately in a cascaded network. The influence of face geometry on the two lighting-dependent intrinsics, diffuse shading and specular reflection, are further eliminated by distributing the intrinsics pixel-wise onto spherical representations using the surface normal as indices. This results in two representations simulating images of a diffuse sphere and a glossy sphere under the input scene lighting. Taking into account the distinctive nature of light sources and ambient terms, we further introduce a two-stage lighting estimator to predict both accurate and realistic lighting from these two representations. Our model is trained supervisedly on a large-scale and high-quality synthetic face image dataset. We demonstrate that our method allows accurate and detailed lighting estimation and intrinsic decomposition, outperforming state-of-the-art methods both qualitatively and quantitatively on real face images.
BibTeX
@article{fei2024split,
author = {Fan Fei and Yean Cheng and Yongjie Zhu and Qian Zheng and Si Li and Gang Pan and Boxin Shi},
title = {{SPLiT: Single Portrait Lighting Estimation via a Tetrad of Face Intrinsics}},
journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
year = {2024},
url = {https://openreview.net/forum?id=doxGg0Pacg},
}Loading related content...