Encoding Auxiliary Information to Restore Compressed Point Cloud Geometry

Gexin Liu, Jiahao Zhu, Dandan Ding, Zhan Ma

Abstract

The standardized Geometry-based Point Cloud Compression (G-PCC) suffers from limited coding performance and low-quality reconstruction. To address this, we propose AuxGR, a performance-complexity tradeoff solution for point cloud geometry restoration: leveraging auxiliary bitstream to enhance the quality of G-PCC compressed point cloud geometry. This auxiliary bitstream efficiently encapsulates spatio-temporal information. For static coding, we perform paired information embedding (PIE) on the G-PCC decoded frame by employing target convolutions from its original counterpart, producing an auxiliary bitstream containing abundant original information. For dynamic coding, in addition to PIE, we propose temporal information embedding (TIE) to capture motion information between the previously restored and the current G-PCC decoded frames. TIE applies target kNN attention between them, which ensures the temporal neighborhood construction for each point and implicitly represents motions. Due to the similarity across temporal frames, only the residuals between TIE and PIE outputs are compressed as auxiliary bitstream. Experimental results demonstrate that AuxGR notably outperforms existing methods in both static and dynamic coding scenarios. Moreover, our framework enables the flexible incorporation of auxiliary information under computation constraints, which is attractive to real applications.

Introduction

Point clouds are essential for representing 3D environments in applications like autonomous driving and virtual reality. However, their massive data volume necessitates efficient compression. The industry standard, Geometry-based Point Cloud Compression (G-PCC), utilizes octree structures for geometry coding. While effective, lossy G-PCC at low bitrates often leads to "point vanishing" (loss of fine details) and "point displacement" (blocky artifacts). Existing deep learning restoration methods often struggle with the irregular motions of dynamic sequences or high computational overhead.

The AuxGR Framework

AuxGR introduces a hybrid restoration framework that uses G-PCC as a base layer and enhances it via a learned auxiliary bitstream. This bitstream provides the necessary spatio-temporal cues to guide the restoration process at the decoder.

The AuxGR framework for (a) static and (b) dynamic point cloud geometry restoration.

Spatio-Temporal Information Embedding

The framework employs two primary embedding strategies:

  • Paired Information Embedding (PIE): In static scenarios, PIE embeds original spatial details into the G-PCC coordinates by comparing the original cloud with a downsampled version.
  • Temporal Information Embedding (TIE): For dynamic sequences, TIE captures motion by analyzing the previously restored frame. Unlike traditional 3D convolutions, TIE utilizes target kNN attention. This allows the model to track points across frames even during rapid motion, effectively constructing temporal neighborhoods.

To optimize efficiency, AuxGR adopts a residual strategy for dynamic coding: only the difference between TIE and PIE outputs is compressed into the auxiliary bitstream, leveraging temporal redundancy.

Restoration and Reconstruction

At the decoder, Spatial Information Aggregation (SIA) uses kNN-based attention to understand local point neighborhoods. Finally, the Coordinate Reconstruction (CR) module restores the geometry using two modes based on cloud density: Probability Prediction for solid clouds and Offset Prediction for sparse clouds to correct point displacements.

Experimental Results

AuxGR was evaluated on standard MPEG datasets, including ShapeNet for static training and 8iVFB for dynamic training.

Performance-complexity tradeoff of AuxGR compared to state-of-the-art methods in static (left) and dynamic (right) scenarios.

Coding Efficiency and Complexity

As shown in the performance-complexity analysis, AuxGR achieves a superior tradeoff compared to existing methods:

  • Static Performance: AuxGR achieves a 53.35% BD-BR reduction over GRASP-Net, a leading competitor.
  • Dynamic Performance: It outperforms the previous state-of-the-art dynamic method by approximately 13-16%.
  • Efficiency: Despite the complexity of kNN attention, the runtime is comparable to the G-PCC base coder and significantly faster than other learned methods (e.g., 65% faster than Fan's and 74% faster than Xia's).

Conclusion

AuxGR provides a flexible and efficient solution for enhancing G-PCC compressed point clouds. By intelligently encoding auxiliary spatio-temporal information, it significantly improves reconstruction quality while maintaining a manageable computational footprint. This framework's flexibility allows it to be toggled based on available bandwidth, making it highly suitable for real-world applications.

Summary updated

Abstract

The standardized Geometry-based Point Cloud Compression (G-PCC) suffers from limited coding performance and low-quality reconstruction. To address this, we propose AuxGR, a performance-complexity tradeoff solution for point cloud geometry restoration: leveraging auxiliary bitstream to enhance the quality of G-PCC compressed point cloud geometry. This auxiliary bitstream efficiently encapsulates spatio-temporal information. For static coding, we perform paired information embedding (PIE) on the G-PCC decoded frame by employing target convolutions from its original counterpart, producing an auxiliary bitstream containing abundant original information. For dynamic coding, in addition to PIE, we propose temporal information embedding (TIE) to capture motion information between the previously restored and the current G-PCC decoded frames. TIE applies target kNN attention between them, which ensures the temporal neighborhood construction for each point and implicitly represents motions. Due to the similarity across temporal frames, only the residuals between TIE and PIE outputs are compressed as auxiliary bitstream. Experimental results demonstrate that AuxGR notably outperforms existing methods in both static and dynamic coding scenarios. Moreover, our framework enables the flexible incorporation of auxiliary information under computation constraints, which is attractive to real applications.
BibTeX
@inproceedings{liu2024encoding,
  author = {Gexin Liu and Jiahao Zhu and Dandan Ding and Zhan Ma},
  title = {{Encoding Auxiliary Information to Restore Compressed Point Cloud Geometry}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=yx2arKnsAl},
}
Figure