Abstract
Due to the diverse sparsity, high dimensionality, and large temporal variation of dynamic point clouds, it remains a challenge to design an efficient point cloud compression method. We propose to code the geometry of a given point cloud by learning a neural volumetric field. Instead of representing the entire point cloud using a single overfit network, we divide the entire space into small cubes and represent each non-empty cube by a neural network and an input latent code. The network is shared among all the cubes in a single frame or multiple frames, to exploit the spatial and temporal redundancy. The neural field representation of the point cloud includes the network parameters and all the latent codes, which are generated by using back-propagation over the network parameters and its input. By considering the entropy of the network parameters and the latent codes as well as the distortion between the original and reconstructed cubes in the loss function, we derive a rate-distortion (R-D) optimal representation. Experimental results show that the proposed coding scheme achieves superior R-D performances compared to the octree-based G-PCC, especially when applied to multiple frames of a point cloud video.
Introduction
Point clouds are essential for 3D applications such as autonomous driving and AR/VR, yet their high dimensionality and sparsity make them difficult to compress. Traditional standards like MPEG G-PCC rely on octree structures to represent 3D space. However, these methods struggle with dynamic point clouds (videos) because the octree structure changes every frame, making motion compensation—the standard technique for exploiting temporal redundancy—extremely difficult. This paper introduces a novel approach that treats a point cloud as a learnable, implicit function through a "neural volumetric field."
Proposed Method
The proposed framework bridges the gap between traditional data structures and modern implicit neural representations. The core innovation lies in a hybrid representation that combines a shallow octree with a shared neural field.
Hybrid Representation and Optimization
The 3D space is first partitioned using a shallow octree (e.g., 5 levels deep) to provide a coarse map of point existence. Each non-empty leaf node, or "cube," is then represented by a small, shared neural network and a specific latent code. Unlike standard deep learning models that use pre-trained weights, this system treats encoding as an optimization task. For a given point cloud, backpropagation is used to find the network parameters ($y$) and latent codes ($z$) that minimize a joint Rate-Distortion (R-D) loss function: $L = Rate + \lambda \cdot Distortion$.
Key Innovations
- Distance-Weighted Focal Loss: To address the extreme sparsity of 3D space, the authors modified the standard focal loss to apply higher penalties to "hallucinated" points far from the actual 3D surface, significantly improving reconstruction quality.
- Amortized Temporal Compression: For point cloud videos, the neural network parameters are shared across multiple frames (e.g., a group of 16 frames). This allows the network to learn common geometric patterns over time, effectively compressing the video by only needing to update the latent codes for each frame, bypassing the need for explicit motion vectors.
- Initialization Separation: To minimize bit-rate, the system encodes only the "delta" from a fixed pseudo-random initialization. Since the delta starts at zero, the entropy remains low, facilitating efficient compression.
Experimental Results
The researchers evaluated the method using the 8i Voxelized Full Bodies (8iVFB) dataset, focusing on four high-quality sequences: Longdress, Loot, Redandblack, and Soldier.
The proposed method consistently outperformed the MPEG G-PCC (TMC13) octree-based codec across all tested static point clouds. The performance gains were even more pronounced for dynamic sequences; by sharing the network over 16 frames, the cost of network parameters is amortized, leading to much higher compression efficiency than coding frames individually.
Conclusion
While the neural volumetric field approach offers superior R-D performance, it introduces new challenges. The encoding process is computationally intensive due to its iterative optimization nature. Furthermore, the current model focuses exclusively on geometry, leaving attribute compression (like color and reflectance) for future work. Despite these limitations, the shift from discrete signal processing to functional representation marks a significant step forward in 3D data compression.
Summary updated