Multiplane NeRF-Supervised Disentanglement of Depth and Camera Pose from Videos
Lists
Only you see this
Abstract
We propose to perform self-supervised disentanglement of depth and camera pose from large-scale videos. We introduce an Autoencoder-based method to reconstruct the input video frames for training, without using any ground-truth annotations of depth and camera. The encoders for our model will estimate the monocular depth and camera pose as the disentangled representations. The decoder will then construct a Multiplane NeRF representation based on the depth encoder feature, and perform rendering to reconstruct the input frames with the estimated camera. The disentanglement is learned with the reconstruction error, based on the assumption that the scene structure does not change in short periods of time in videos. Once the model is learned, it can be applied to multiple applications including depth estimation, camera pose estimation, and single image novel view synthesis. We show substantial improvements over previous self-supervised approaches on all tasks and even better results than counterparts trained with camera ground-truths in some applications. Our code will be made publicly available.
BibTeX
@misc{fu2022multiplane,
author = {Yang Fu and Ishan Misra and Xiaolong Wang},
title = {{Multiplane NeRF-Supervised Disentanglement of Depth and Camera Pose from Videos}},
year = {2022},
url = {https://openreview.net/forum?id=uefYIUDf95},
}Loading related content...