Abstract
3D point cloud compression is essential for emerging applications in virtual reality, augmented reality, and autonomous driving. While the industry-standard Geometry-based Point Cloud Compression (G-PCC) provides effective data reduction, it often introduces significant geometric artifacts, such as point vanishing and displacement. We propose AuxGR, a novel hybrid restoration framework that utilizes a learned auxiliary bitstream to guide the decoder in recovering high-fidelity geometry. By integrating spatial and temporal embedding modules—specifically a k-Nearest Neighbors (kNN) attention mechanism for dynamic sequences—AuxGR effectively handles large-scale motion and diverse point cloud densities. Experimental results demonstrate that AuxGR achieves state-of-the-art performance, reducing BD-BR by over 50% for static solid point clouds and up to 25% for dynamic sequences, while maintaining a low computational footprint suitable for real-time deployment.
Introduction
Point clouds, which consist of massive collections of 3D coordinates, are the backbone of modern immersive media and spatial sensing. However, their raw data volume is immense, necessitating lossy compression standards like G-PCC. These standards often result in two primary types of distortion: point vanishing, where points are entirely lost during quantization, and point displacement, where points shift from their original positions.
While deep learning has shown promise in post-processing these artifacts, existing methods often struggle with high computational complexity or fail to leverage temporal information in moving sequences. Furthermore, traditional convolutional approaches are frequently inadequate for dynamic point clouds where rapid motion causes points to "jump" outside fixed receptive fields. AuxGR addresses these challenges by treating G-PCC as a base layer and adding a learned enhancement layer guided by a lightweight auxiliary bitstream.
The AuxGR Framework
The core innovation of AuxGR is the generation of an auxiliary bitstream at the encoder that provides critical geometric cues to the decoder. This bitstream is extremely compact, typically requiring less than 0.1 bits per point (bpp), yet it significantly improves restoration quality.
The framework consists of several key components:
- Paired Information Embedding (PIE): Used in static modes to compare original and distorted frames, embedding missing geometric details into the auxiliary stream.
- Temporal Information Embedding (TIE): In dynamic sequences, TIE captures motion between the current frame and the previously restored one. To overcome the limitations of standard convolutions, TIE utilizes target kNN (k-Nearest Neighbors) attention, allowing the model to find matching points even during fast motion.
- Spatial Information Aggregation (SIA): A decoder-side module that cleans up spatial noise by analyzing a point's local neighborhood.
- Coordinate Reconstruction (CR): A flexible output module with two modes: Probability Generation for solid objects (to combat vanishing) and Offset Prediction for dense or sparse scans (to fix displacement).
Overcoming Motion with kNN Attention
A significant hurdle in dynamic point cloud restoration is the irregular and rapid movement of points across frames. Standard 3D convolutions rely on fixed kernels that may miss relevant points if they move too far between time steps. AuxGR’s TIE module replaces these convolutions with a search-based kNN attention mechanism. This ensures that the model always identifies a local neighborhood for every point, regardless of the magnitude of motion, making it robust for complex dynamic content like the "Dancer" or "Basketball" sequences.
Experimental Results
AuxGR was evaluated using a wide range of datasets, including 20,000 meshes from ShapeNet for static training and the 8i Voxelized Full Bodies (8iVFB) dataset for dynamic training. Testing was conducted on MPEG-recommended benchmarks and the Owlii dynamic dataset.
As shown in the performance-complexity analysis, AuxGR establishes a new state-of-the-art (SOTA):
- Static Performance: Achieved over 50% BD-BR reduction compared to GRASP-Net for solid point clouds.
- Dynamic Performance: Outperformed previous SOTA methods by 13% to 25% in BD-BR reduction for moving sequences.
- Efficiency: AuxGR offers a superior tradeoff, running significantly faster than competing methods like Fan’s or Xia’s while delivering higher quality restoration.
Conclusion
AuxGR demonstrates that a small amount of guided auxiliary data can lead to massive gains in point cloud restoration quality. By combining kNN-based temporal attention with density-aware reconstruction, the framework provides a versatile and efficient solution for both static and dynamic 3D content. Future work will explore extending this framework to attribute compression, such as restoring color and reflectance data alongside geometry.
Summary updated