PhySG: Inverse Rendering With Spherical Gaussians for Physics-Based Material Editing and Relighting

Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, Noah Snavely

Abstract

We present an end-to-end inverse rendering pipeline that includes a fully differentiable renderer, and can reconstruct geometry, materials, and illumination from scratch from a set of images. Our rendering framework represents specular BRDFs and environmental illumination using mixtures of spherical Gaussians, and represents geometry as a signed distance function parameterized as a Multi-Layer Perceptron. The use of spherical Gaussians allows us to efficiently solve for approximate light transport, and our method works on scenes with challenging non-Lambertian reflectance captured under natural, static illumination. We demonstrate, with both synthetic and real data, that our reconstruction not only can render novel viewpoints, but also enables physics-based appearance editing of materials and illumination.
BibTeX
@inproceedings{zhang2021physg,
  author = {Kai Zhang and Fujun Luan and Qianqian Wang and Kavita Bala and Noah Snavely},
  title = {{PhySG: Inverse Rendering With Spherical Gaussians for Physics-Based Material Editing and Relighting}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  url = {https://openreview.net/forum?id=qmarwWJ2cO},
}
Figure