Interpreting Latent Representation in Neural Radiance Fields for Manipulating Object Semantics
Lists
Only you see this
Abstract
Manipulating 3D objects has been among the active research topic for 3D vision. With the development and success of neural radiance field (NeRF) [1] on scene modeling, synthesizing and manipulating 3D objects using such a representation becomes desirable. In this paper, we introduce a semantic-aware generative NeRF, which is able to interpret the latent representation learned by category-specific generative NeRFs and to achieve editing of particular part attributes. With pretrained generative NeRF, we propose to deploy a semantic segmentor for performing part segmentation on the object category. This allows the rendering of the 2D image and prediction of the corresponding segmentation mask. Our proposed scheme learns to manipulate the resulting latent representation, optimized to edit the object part of interest with varying degrees. We conduct experiments on various object categories on benchmark datasets, and the results successfully verify the effectiveness and practicality of our proposed model.
BibTeX
@inproceedings{huang2023interpreting,
author = {Yu-Shan Huang and Sheng-Yu Huang and Hao-Yu Hsu and Yu-Chiang Frank Wang},
title = {{Interpreting Latent Representation in Neural Radiance Fields for Manipulating Object Semantics}},
booktitle = {ICIP},
year = {2023},
doi = {10.1109/icip49359.2023.10222650},
}Loading related content...