PCQM: A Full-Reference Quality Metric for Colored 3D Point Clouds

Gabriel Meynet, Yana Nehmé, Julie Digne, Guillaume Lavoué

TL;DR

This paper introduces PCQM, a novel objective metric for assessing the visual quality of colored 3D point clouds. By combining geometry and color features, PCQM significantly outperforms existing methods in predicting human perception of quality, paving the way for better compression and rendering of 3D content.

What Problem Does This Paper Address?

3D point clouds are becoming increasingly prevalent in various applications, from gaming and virtual reality to industrial design. However, their large data size necessitates efficient compression techniques. Lossy compression inevitably degrades the quality of these point clouds, impacting their visual fidelity. To optimize compression algorithms and ensure a good user experience, we need objective metrics that can accurately predict how humans perceive the quality of these distorted point clouds. While metrics exist for 2D images and even for the geometry of 3D point clouds, there's a significant gap in evaluating the combined visual quality of colored 3D point clouds. This paper aims to fill that gap.

Methods

The core of PCQM is a data-driven approach that combines perceptual features related to both geometry and color.

  • Point-to-Surface Correspondence: To compare a distorted point cloud (D) with a reference point cloud (R), they first establish a correspondence. Instead of simple nearest-neighbor matching, they fit a local quadric surface to the neighbors of a reference point to find its projected point on the distorted surface. This provides a more robust correspondence.
  • Geometry-Based Features: They leverage curvature-based features, similar to those used in the PC-MSDM metric. These features capture how the shape of the point cloud is distorted.
  • Color-Based Features: This is a key novelty. They extend existing 2D image color quality features to the 3D point cloud domain. They convert RGB colors to the perceptually uniform LAB2000HL color space and extract features related to lightness, chroma, and hue comparisons and contrasts.
  • Optimal Feature Combination: The individual geometry and color features are then combined using logistic regression. They perform an exhaustive search to find the optimal subset of features and their corresponding weights that best predict subjective quality scores. This results in a linear model that aggregates the selected features.

Datasets

The primary dataset used for training and evaluation is the IST Rendering Point Cloud Dataset [8]. This dataset consists of 54 colored point clouds derived from 6 reference models, compressed using three different codecs (Octree-pruning, MPEG G-PCC, and MPEG V-PCC) at various quality levels. Subjective Mean Opinion Scores (MOS) were collected using a DSIS (Double Stimulus Impairment Scale) methodology from 20 subjects for each rendered point cloud. The authors specifically considered the MOS values from the session where point clouds were rendered with colors.

Key Insights and Ideas

  • Color Matters Significantly: The study highlights that color information plays a crucial role in the human perception of 3D point cloud quality, often as much as, or even more than, geometric distortions.
  • Complementary Nature of Features: Neither geometry nor color features alone are sufficient for accurate quality prediction. The optimal metric PCQM effectively combines both, demonstrating their complementary nature in how humans perceive visual quality.
  • Lightness is Key for Color: Among the color features, those related to lightness (comparison, contrast, and structure) showed the strongest correlation with subjective scores, suggesting that variations in brightness are highly perceptible.
  • Robust Correspondence is Crucial: The point-to-surface correspondence mechanism, which goes beyond simple nearest-neighbor matching, is vital for accurately capturing geometric distortions.

Key Results

  • Outperforms State-of-the-Art: PCQM achieved the highest correlation with subjective MOS scores, outperforming existing metrics, including geometry-only metrics and simple distance-based measures.
  • Best Model Composition: The optimized PCQM model is a linear combination of three features: Curvature structure (f3), Lightness comparison (f4), and Lightness structure (f6).
  • Quantitative Performance: On the IST dataset, PCQM achieved a Pearson correlation of 0.90 and a Spearman correlation of 0.83 when evaluated over the entire dataset. When averaged over 20 test sets, it achieved a Pearson correlation of 0.873 and a Spearman correlation of 0.807.
  • Code Availability: The authors have made the source code for PCQM publicly available, which is a significant contribution to the research community.

Limitations and Future Work

  • Limited Dataset: While the IST dataset is comprehensive, the authors acknowledge that it's based on a specific set of reference models and compression techniques. Future work could involve evaluating PCQM on a wider variety of datasets and compression methods.
  • No-Reference Metric: PCQM is a full-reference metric, meaning it requires access to the original, undistorted point cloud for comparison. Developing a no-reference metric that can predict quality without the original would be a valuable next step.
  • Generalizability: While the current results are promising, further validation on diverse 3D content and distortion types would be beneficial to confirm the generalizability of the proposed metric.
  • Computational Cost: While the paper focuses on accuracy, future work could also explore optimizations for real-time applications if computational efficiency becomes a bottleneck.

Summary updated

Abstract

3D point clouds constitute an emerging multimedia content, now used in a wide range of applications. The main drawback of this representation is the size of the data since typical point clouds may contain millions of points, usually associated with both geometry and color information. Consequently, a significant amount of work has been devoted to the efficient compression of this representation. Lossy compression leads to a degradation of the data and thus impacts the visual quality of the displayed content. In that context, predicting perceived visual quality computationally is essential for the optimization and evaluation of compression algorithms. In this paper, we introduce PCQM, a full-reference objective metric for visual quality assessment of 3D point clouds. The metric is an optimally-weighted linear combination of geometry-based and color-based features. We evaluate its performance on an open subjective dataset of colored point clouds compressed by several algorithms; the proposed quality assessment approach outperforms all previous metrics in terms of correlation with mean opinion scores.
BibTeX
@inproceedings{meynet2020pcqm,
  author = {Gabriel Meynet and Yana Nehmé and Julie Digne and Guillaume Lavoué},
  title = {{PCQM: A Full-Reference Quality Metric for Colored 3D Point Clouds}},
  booktitle = {QoMEX},
  year = {2020},
  url = {https://openreview.net/forum?id=eV0syGqNuz},
}
Figure