Learning to Predict on Octree for Scalable Point Cloud Geometry Coding

Yixiang Mao, Yueyu Hu, Yao Wang

Abstract

Octree-based point cloud representation and compression have been adopted by the MPEG G-PCC standard. However, it only uses handcrafted methods to predict the probability that a leaf node is non-empty, which is then used for entropy coding. We propose a novel approach for predicting such probabilities for geometry coding, which applies a denoising neural network to a "noisy" context cube that includes both neighboring decoded voxels as well as uncoded voxels. We further propose a convolution-based model to upsample the decoded point cloud at a coarse resolution on the decoder side. Integration of the two approaches significantly improves the rate-distortion performance for geometry coding compared to the original G-PCC standard and other baseline methods for dense point clouds. The proposed octree-based entropy coding approach is naturally scalable, which is desirable for dynamic rate adaptation in point cloud streaming systems.

Introduction

Point clouds—collections of thousands or millions of 3D points—are the backbone of VR, AR, and autonomous driving. However, they are notoriously data-heavy. To manage this, the industry standard (MPEG G-PCC) uses an octree structure, which recursively divides 3D space into smaller cubes (voxels).

The efficiency of this compression relies on entropy coding: predicting whether a specific sub-cube is occupied or empty. Currently, G-PCC uses handcrafted context tables for these predictions, which are often suboptimal. Furthermore, existing machine learning attempts at this problem often struggle with "scalability"—the ability to stop decoding at a lower resolution and still get a usable 3D model—which is essential for streaming video over fluctuating internet connections.

Methodology

The authors propose a two-part solution involving conditional probability estimation and resolution enhancement.

Conditional Probability Estimation via Denoising

When coding a specific level of the octree, some neighboring nodes are already decoded (known), while others are not. The authors create a "noisy" context cube where known occupied/empty nodes are marked 1 or 0, and unknown nodes (whose parents are occupied) are assigned a value of 0.5. A 3D convolutional neural network (CNN) then "denoises" this cube to predict the actual occupancy probabilities for the current nodes.

Two architectures were developed: Model A (fully convolutional for maximum accuracy) and Model B (a hybrid of convolution and fully connected layers designed for significantly lower computational complexity).

Architectures for probability estimation: (a) Model A (fully convolutional) and (b) Model B (hybrid)

Resolution Enhancement (Upsampling)

If a user has low bandwidth, the bitstream might only contain data for a coarse version of the point cloud. The authors provide a 3D CNN at the decoder side that can "guess" the finer details, upsampling a coarse point cloud to a higher resolution without requiring any additional bits to be sent over the wire.

Experimental Results

The models were trained on a subset of ShapeNetCore (over 51,000 3D models) and evaluated on the 8iVSLF dataset, the standard benchmark for dense point cloud compression featuring high-quality captures like "Longdress" and "Soldier."

  • Lossless Compression: Compared to the MPEG G-PCC standard, the proposed method (Model A) reduces the bit rate by an average of 32.1%.
  • Lossy Compression: When combined with the upsampling model, the method saves roughly 80% of bits to achieve the same reconstruction quality as G-PCC.
  • Visual Quality: The upsampling model significantly reduces the "blocky" look of low-bitrate point clouds, producing much smoother and more recognizable 3D surfaces.
Visual comparison of reconstruction quality and Rate-Distortion curves across standard datasets

Conclusion

By framing octree occupancy prediction as a denoising task and integrating a decoder-side upsampling model, this work significantly advances point cloud compression. The approach is inherently scalable, making it ideal for dynamic rate adaptation in streaming systems. While 3D convolutions remain computationally expensive, the development of lower-complexity variants like Model B (which reduced FLOPs by 85%) paves the way for real-time hardware adoption. Future work will focus on universal upsampling models and extending the framework to sparse LiDAR point clouds.

Summary updated

Abstract

Octree-based point cloud representation and compression have been adopted by the MPEG G-PCC standard. However, it only uses handcrafted methods to predict the probability that a leaf node is non-empty, which is then used for entropy coding. We propose a novel approach for predicting such probabilities for geometry coding, which applies a denoising neural network to a "noisy" context cube that includes both neighboring decoded voxels as well as uncoded voxels. We further propose a convolution-based model to upsample the decoded point cloud at a coarse resolution on the decoder side. Integration of the two approaches significantly improves the rate-distortion performance for geometry coding compared to the original G-PCC standard and other baseline methods for dense point clouds. The proposed octree-based entropy coding approach is naturally scalable, which is desirable for dynamic rate adaptation in point cloud streaming systems.
BibTeX
@misc{mao2022learning,
  author = {Yixiang Mao and Yueyu Hu and Yao Wang},
  title = {{Learning to Predict on Octree for Scalable Point Cloud Geometry Coding}},
  year = {2022},
  eprint = {2209.02226},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.02226},
}
Figure