Abstract
Point cloud data is essential for immersive media but requires massive bandwidth. The Video-based Point Cloud Compression (V-PCC) standard addresses this by projecting 3D data into 2D video frames. However, standard video encoders waste bits on unoccupied background pixels. This paper proposes an occupancy-map-based Rate Distortion Optimization (RDO) scheme that masks distortion in these empty areas. By focusing the bit budget on occupied pixels, the method achieves significant bitrate savings—up to 15.4%—without compromising 3D reconstruction quality.
Introduction
Point clouds—collections of thousands or millions of 3D points—are essential for VR, AR, and telepresence, but they are notoriously data-heavy. To compress them, the state-of-the-art Video-based Point Cloud Compression (V-PCC) standard projects 3D points onto 2D "patches" and organizes them into a standard video frame. This allows the use of powerful video codecs like HEVC (H.265).
However, these 2D frames contain many "unoccupied" pixels—the gaps between the patches. Standard video encoders treat every pixel as equally important, spending a significant portion of the "bit budget" to accurately reconstruct these empty background pixels, even though they are discarded when the 3D object is rebuilt. This represents a substantial waste of data.
Proposed Methodology
The authors propose an occupancy-map-based RDO scheme. In standard video encoding, the encoder makes decisions (such as motion vector selection) based on a cost function ($J$):
$$J = D + \lambda R$$
Where $D$ is distortion (error) and $R$ is the bitrate (cost in bits). The researchers modified this formula by introducing a distortion mask ($M$):
$$J = \sum (D_i \times M_i) + \lambda R$$
The mask $M$ is derived from the point cloud's "occupancy map"—metadata that identifies which pixels are part of the object. If a pixel is occupied, $M=1$; if unoccupied, $M=0$. By setting the distortion of empty pixels to zero, the encoder focuses solely on minimizing the bitrate in those areas, choosing the most efficient encoding modes (like skip modes or zero motion vectors). This logic was applied to three critical parts of the encoding process: Intra-frame prediction, Inter-frame (motion) prediction, and Sample Adaptive Offset (SAO) filtering.
Experimental Results
The algorithm was evaluated using five standard Dynamic Point Cloud (DPC) sequences: Loot, RedAndBlack (RAB), Soldier, Queen, and LongDress (LD). The proposed method showed impressive gains, particularly in video sequences with motion.
In Random Access configurations, the method achieved an average of 11.9% savings for geometry and 15.4% savings for attributes. In All Intra configurations, gains were 2.9% for geometry and 1.1% for attributes. Crucially, the method reduces encoding and decoding complexity because the encoder chooses simpler modes for unoccupied areas, and visual fidelity remains nearly identical to the high-bitrate anchor.
Conclusion
This occupancy-aware approach leverages existing metadata to provide a "pure" encoder-side improvement that requires zero extra data to be sent to the decoder. Future work will focus on refining the handling of blocks on patch boundaries, which contain a mix of occupied and unoccupied pixels, to further enhance compression efficiency.
Summary updated