Material Swapping for 3D Scenes using a Learnt Material Similarity Measure

Maxine Perroni-Scharf, Kalyan Sunkavalli, Jonathan Eisenmann, Yannick Hold-GeoffroyWorkshops @ CVPR 2022 · OpenReview

Abstract

We present a method for augmenting photo-realistic 3D scene assets by automatically recognizing, matching, and swapping their materials. Our method proposes a material matching pipeline for the efficient replacement of unknown materials with perceptually similar PBR materials from a database, enabling the quick creation of many variations of a given 3D synthetic scene. At the heart of this method is a novel material similarity feature that is learnt, in conjunction with optimal lighting conditions, by fine-tuning a deep neural network on a material classification task using our proposed dataset. Our evaluation demonstrates that lighting optimization improves CNN-based texture feature extraction methods and better estimates material properties. We conduct a series of experiments showing our method’s ability to augment photo-realistic indoor scenes using both standard and procedurally generated PBR materials.
BibTeX
@inproceedings{perronischarf2022material,
  author = {Maxine Perroni-Scharf and Kalyan Sunkavalli and Jonathan Eisenmann and Yannick Hold-Geoffroy},
  title = {{Material Swapping for 3D Scenes using a Learnt Material Similarity Measure}},
  booktitle = {Workshops @ CVPR},
  year = {2022},
  url = {https://openreview.net/forum?id=o_hNIGNHfVx},
}

Loading related content...

Figure