Occupancy-Map-Based Rate Distortion Optimization for Video-Based Point Cloud Compression

Li Li 0040, Zhu Li 0001, Shan Liu, Houqiang Li

Abstract

Point cloud data is essential for immersive media but requires massive bandwidth. The Video-based Point Cloud Compression (V-PCC) standard addresses this by projecting 3D data into 2D video frames. However, standard video encoders waste bits on unoccupied background pixels. This paper proposes an occupancy-map-based Rate Distortion Optimization (RDO) scheme that masks distortion in these empty areas. By focusing the bit budget on occupied pixels, the method achieves significant bitrate savings—up to 15.4%—without compromising 3D reconstruction quality.

Introduction

Point clouds—collections of thousands or millions of 3D points—are essential for VR, AR, and telepresence, but they are notoriously data-heavy. To compress them, the state-of-the-art Video-based Point Cloud Compression (V-PCC) standard projects 3D points onto 2D "patches" and organizes them into a standard video frame. This allows the use of powerful video codecs like HEVC (H.265).

However, these 2D frames contain many "unoccupied" pixels—the gaps between the patches. Standard video encoders treat every pixel as equally important, spending a significant portion of the "bit budget" to accurately reconstruct these empty background pixels, even though they are discarded when the 3D object is rebuilt. This represents a substantial waste of data.

2D projection of 3D point cloud patches. The green areas represent unoccupied pixels that are typically discarded during reconstruction.

Proposed Methodology

The authors propose an occupancy-map-based RDO scheme. In standard video encoding, the encoder makes decisions (such as motion vector selection) based on a cost function ($J$):

$$J = D + \lambda R$$

Where $D$ is distortion (error) and $R$ is the bitrate (cost in bits). The researchers modified this formula by introducing a distortion mask ($M$):

$$J = \sum (D_i \times M_i) + \lambda R$$

The mask $M$ is derived from the point cloud's "occupancy map"—metadata that identifies which pixels are part of the object. If a pixel is occupied, $M=1$; if unoccupied, $M=0$. By setting the distortion of empty pixels to zero, the encoder focuses solely on minimizing the bitrate in those areas, choosing the most efficient encoding modes (like skip modes or zero motion vectors). This logic was applied to three critical parts of the encoding process: Intra-frame prediction, Inter-frame (motion) prediction, and Sample Adaptive Offset (SAO) filtering.

Experimental Results

The algorithm was evaluated using five standard Dynamic Point Cloud (DPC) sequences: Loot, RedAndBlack (RAB), Soldier, Queen, and LongDress (LD). The proposed method showed impressive gains, particularly in video sequences with motion.

Rate-distortion curves for the 'Loot' sequence, comparing the proposed occupancy-map-based RDO against the anchor V-PCC implementation for both geometry (D1) and attributes (Luma).

In Random Access configurations, the method achieved an average of 11.9% savings for geometry and 15.4% savings for attributes. In All Intra configurations, gains were 2.9% for geometry and 1.1% for attributes. Crucially, the method reduces encoding and decoding complexity because the encoder chooses simpler modes for unoccupied areas, and visual fidelity remains nearly identical to the high-bitrate anchor.

Conclusion

This occupancy-aware approach leverages existing metadata to provide a "pure" encoder-side improvement that requires zero extra data to be sent to the decoder. Future work will focus on refining the handling of blocks on patch boundaries, which contain a mix of occupied and unoccupied pixels, to further enhance compression efficiency.

Summary updated

BibTeX
@inproceedings{li2019occupancy,
  author = {Li Li and Zhu Li and Shan Liu and Houqiang Li},
  title = {{Occupancy-Map-Based Rate Distortion Optimization for Video-Based Point Cloud Compression}},
  booktitle = {ICIP},
  year = {2019},
  doi = {10.1109/icip.2019.8803233},
}
Figure