JOINTRF: End-To-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression

Zihan Zheng, Houqiang Zhong, Qiang Hu, Xiaoyun Zhang, Li Song, Ya Zhang, Yanfeng Wang

Abstract

Neural Radiance Field (NeRF) excels in photo-realistically static scenes, inspiring numerous efforts to facilitate volumetric videos. However, rendering dynamic and long-sequence radiance fields remains challenging due to the significant data required to represent volumetric videos. In this paper, we propose a novel end-to-end joint optimization scheme of dynamic NeRF representation and compression, called JointRF, thus achieving significantly improved quality and compression efficiency against the previous methods. Specifically, JointRF employs a compact residual feature grid and a coefficient feature grid to represent the dynamic NeRF. This representation handles large motions without compromising quality while concurrently diminishing temporal redundancy. We also introduce a sequential feature compression subnetwork to further reduce spatial-temporal redundancy. Finally, the representation and compression subnetworks are end-to-end trained combined within the JointRF. Extensive experiments demonstrate that JointRF can achieve superior compression performance across various datasets.
BibTeX
@inproceedings{zheng2024jointrf,
  author = {Zihan Zheng and Houqiang Zhong and Qiang Hu and Xiaoyun Zhang and Li Song and Ya Zhang and Yanfeng Wang},
  title = {{JOINTRF: End-To-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression}},
  booktitle = {ICIP},
  year = {2024},
  doi = {10.1109/icip51287.2024.10647336},
}

Loading related content...

Figure