Video-based Point Cloud Compression Artifact Removal

Anique Akhtar, Wen Gao, Li Li, Wei Jia, Shan Liu

Abstract

Photo-realistic point cloud capture and transmission are fundamental enablers for immersive visual communication. The Video-based Point Cloud Compression (V-PCC) standard developed by MPEG delivers state-of-the-art efficiency but introduces quantization errors that manifest as visually jarring geometry artifacts. We propose a novel out-of-the-loop artifact removal framework that significantly improves reconstruction quality without increasing bandwidth. Our solution employs a cube-based sampling scheme, a 3D sparse convolutional network, and an aggregation strategy. By leveraging the prior knowledge that V-PCC quantization noise is strictly directional, our method achieves an average BD-Rate saving of 11.3% and substantial gains in objective PSNR.

Introduction: The Challenge of Point Cloud Compression

Dynamic 3D point clouds are essential for applications ranging from AR/VR to volumetric telepresence. However, the massive data requirements of these clouds make efficient compression critical. The MPEG V-PCC standard addresses this by projecting 3D patches onto 2D planes and encoding them using standard 2D video codecs like HEVC.

Visual artifacts on a point cloud reconstruction after V-PCC compression.

While highly effective at reducing data size, V-PCC's lossy quantization process introduces severe "blocking" artifacts in the 3D geometry, especially at low bitrates. Furthermore, because lossy compression alters the total number of points in the cloud, standard 3D denoising techniques struggle to map noisy compressed points back to the clean ground-truth geometry. This paper introduces a post-processing framework designed to clean these artifacts.

Methodology

The proposed framework consists of three main stages: sampling, artifact removal, and aggregation. This out-of-the-loop approach allows for quality enhancement without modifying the underlying V-PCC codec.

Overview of the proposed artifact removal framework pipeline.

Cube-based Sampling

To address the "patch correspondence mismatch" caused by varying point counts, we avoid traditional k-Nearest Neighbors (k-NN) grouping. Instead, we use Farthest Point Sampling (FPS) to select center points and extract all points within a fixed 3D cube volume. This ensures that the network compares consistent physical areas across different compression levels.

3D Sparse Artifact Removal Network

The extracted patches are processed by a 3D U-Net built with submanifold sparse convolutions. This architecture efficiently handles the variable number of points in each cube without "dilating" the sparse data, which saves significant memory and computational resources.

Directional Noise Prior

A key insight of this work is the realization that V-PCC geometry distortion is highly constrained. Because points are projected orthogonally onto bounding box planes, quantization noise always falls strictly along a single spatial axis (x, y, or z). The network exploits this by predicting a 3D projection vector (converted to a one-hot directional vector) and a 1D scalar weight, rather than attempting to predict raw 3D noise directly.

Experimental Results

The framework was evaluated on the 8i voxelized full bodies dataset, using sequences such as redandblack, soldier, and queen for testing.

  • Bandwidth Savings: The method achieves an impressive 11.3% BD-Rate savings, delivering the same visual quality at a significantly lower bitrate.
  • PSNR Improvements: At the lowest bitrates, the method improved the point-to-point MSE PSNR by an average of 0.85 dB.
  • Visual Quality: Qualitative results show a dramatic reduction in blocking artifacts, particularly on the surfaces of human models, and the successful reconstruction of geometry lost during encoding.

Conclusion

This paper presents the first deep learning-based framework for removing geometry artifacts from V-PCC encoded point clouds. By incorporating domain-specific priors about quantization noise and utilizing efficient sparse convolutions, the proposed method offers a powerful tool for enhancing immersive visual communication. Future work will extend these techniques to address texture artifacts and explore tighter integration with the V-PCC decoding process.

Summary updated

Abstract

Photo-realistic point cloud capture and transmission are the fundamental enablers for immersive visual communication. The coding process of dynamic point clouds, especially video-based point cloud compression (V-PCC) developed by the MPEG standardization group, is now delivering state-of-the-art performance in compression efficiency. V-PCC is based on the projection of the point cloud patches to 2D planes and encoding the sequence as 2D texture and geometry patch sequences. However, the resulting quantization errors from coding can introduce compression artifacts, which can be very unpleasant for the quality of experience (QoE). In this work, we developed a novel out-of-the-loop point cloud geometry artifact removal solution that can significantly improve reconstruction quality without additional bandwidth cost. Our novel framework consists of a point cloud sampling scheme, an artifact removal network, and an aggregation scheme. The point cloud sampling scheme employs a cube-based neighborhood patch extraction to divide the point cloud into patches. The geometry artifact removal network then processes these patches to obtain artifact-removed patches. The artifact-removed patches are then merged together using an aggregation scheme to obtain the final artifact-removed point cloud. We employ 3D deep convolutional feature learning for geometry artifact removal that jointly recovers both the quantization direction and the quantization noise level by exploiting projection and quantization prior. The simulation results demonstrate that the proposed method is highly effective and can considerably improve the quality of the reconstructed point cloud.
BibTeX
@article{akhtar2022video,
  author = {Anique Akhtar and Wen Gao and Li Li and Zhu Li and Wei Jia and Shan Liu},
  title = {{Video-based Point Cloud Compression Artifact Removal}},
  journal = {IEEE Trans. Multimedia},
  year = {2022},
  doi = {10.1109/tmm.2021.3090148},
}
Figure