Abstract
Photo-realistic point cloud capture and transmission are fundamental enablers for immersive visual communication. The Video-based Point Cloud Compression (V-PCC) standard developed by MPEG delivers state-of-the-art efficiency but introduces quantization errors that manifest as visually jarring geometry artifacts. We propose a novel out-of-the-loop artifact removal framework that significantly improves reconstruction quality without increasing bandwidth. Our solution employs a cube-based sampling scheme, a 3D sparse convolutional network, and an aggregation strategy. By leveraging the prior knowledge that V-PCC quantization noise is strictly directional, our method achieves an average BD-Rate saving of 11.3% and substantial gains in objective PSNR.
Introduction: The Challenge of Point Cloud Compression
Dynamic 3D point clouds are essential for applications ranging from AR/VR to volumetric telepresence. However, the massive data requirements of these clouds make efficient compression critical. The MPEG V-PCC standard addresses this by projecting 3D patches onto 2D planes and encoding them using standard 2D video codecs like HEVC.
While highly effective at reducing data size, V-PCC's lossy quantization process introduces severe "blocking" artifacts in the 3D geometry, especially at low bitrates. Furthermore, because lossy compression alters the total number of points in the cloud, standard 3D denoising techniques struggle to map noisy compressed points back to the clean ground-truth geometry. This paper introduces a post-processing framework designed to clean these artifacts.
Methodology
The proposed framework consists of three main stages: sampling, artifact removal, and aggregation. This out-of-the-loop approach allows for quality enhancement without modifying the underlying V-PCC codec.
Cube-based Sampling
To address the "patch correspondence mismatch" caused by varying point counts, we avoid traditional k-Nearest Neighbors (k-NN) grouping. Instead, we use Farthest Point Sampling (FPS) to select center points and extract all points within a fixed 3D cube volume. This ensures that the network compares consistent physical areas across different compression levels.
3D Sparse Artifact Removal Network
The extracted patches are processed by a 3D U-Net built with submanifold sparse convolutions. This architecture efficiently handles the variable number of points in each cube without "dilating" the sparse data, which saves significant memory and computational resources.
Directional Noise Prior
A key insight of this work is the realization that V-PCC geometry distortion is highly constrained. Because points are projected orthogonally onto bounding box planes, quantization noise always falls strictly along a single spatial axis (x, y, or z). The network exploits this by predicting a 3D projection vector (converted to a one-hot directional vector) and a 1D scalar weight, rather than attempting to predict raw 3D noise directly.
Experimental Results
The framework was evaluated on the 8i voxelized full bodies dataset, using sequences such as redandblack, soldier, and queen for testing.
- Bandwidth Savings: The method achieves an impressive 11.3% BD-Rate savings, delivering the same visual quality at a significantly lower bitrate.
- PSNR Improvements: At the lowest bitrates, the method improved the point-to-point MSE PSNR by an average of 0.85 dB.
- Visual Quality: Qualitative results show a dramatic reduction in blocking artifacts, particularly on the surfaces of human models, and the successful reconstruction of geometry lost during encoding.
Conclusion
This paper presents the first deep learning-based framework for removing geometry artifacts from V-PCC encoded point clouds. By incorporating domain-specific priors about quantization noise and utilizing efficient sparse convolutions, the proposed method offers a powerful tool for enhancing immersive visual communication. Future work will extend these techniques to address texture artifacts and explore tighter integration with the V-PCC decoding process.
Summary updated