FV-NeRV: Neural Compression for Free Viewpoint Videos

Takuya Fujihashi, Sorachi Kato, Toshiaki Koike-AkinoSubmission to ICLR 2025 · Withdrawn

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

The delivery of free viewpoint videos (FVVs) is gaining popularity because of their ability to provide freely switchable perspectives to remote users as immersive experiences. While smooth view switching is crucial for enhancing user's experiences, FVV delivery faces a significant challenge in balancing traffic and decoding latency. The typical approach sends limited viewpoints and synthesizes the remainings on the user, reducing traffic, but increasing decoding delays. Alternatively, sending more viewpoints reduces the delay, but requires more bandwidth for transmission. In this paper, we propose a novel FVV representation format, Free Viewpoint-Neural Representation for Videos~(FV-NeRV), to address this dilemma in FVV delivery. FV-NeRV reduces both traffic and decoding delay even for content with a large number of virtual viewpoints by overfitting compact neural networks to all viewpoints and pruning and quantizing the trained model. Experiments show that FV-NeRV achieves a comparable or even superior traffic reduction with a faster decoding speed compared to existing FVV codecs.
BibTeX
@inproceedings{fujihashi2024fv,
  author = {Takuya Fujihashi and Sorachi Kato and Toshiaki Koike-Akino},
  title = {{FV-NeRV: Neural Compression for Free Viewpoint Videos}},
  booktitle = {Compression @ NeurIPS},
  year = {2024},
  url = {https://openreview.net/forum?id=fDEKEGqbLw},
}

Loading related content...

Figure