TL;DR
This paper introduces PCQM, a novel objective metric for assessing the visual quality of colored 3D point clouds. By combining geometry and color features, PCQM significantly outperforms existing methods in predicting human perception of quality, paving the way for better compression and rendering of 3D content.
What Problem Does This Paper Address?
3D point clouds are becoming increasingly prevalent in various applications, from gaming and virtual reality to industrial design. However, their large data size necessitates efficient compression techniques. Lossy compression inevitably degrades the quality of these point clouds, impacting their visual fidelity. To optimize compression algorithms and ensure a good user experience, we need objective metrics that can accurately predict how humans perceive the quality of these distorted point clouds. While metrics exist for 2D images and even for the geometry of 3D point clouds, there's a significant gap in evaluating the combined visual quality of colored 3D point clouds. This paper aims to fill that gap.
Methods
The core of PCQM is a data-driven approach that combines perceptual features related to both geometry and color.
- Point-to-Surface Correspondence: To compare a distorted point cloud (D) with a reference point cloud (R), they first establish a correspondence. Instead of simple nearest-neighbor matching, they fit a local quadric surface to the neighbors of a reference point to find its projected point on the distorted surface. This provides a more robust correspondence.
- Geometry-Based Features: They leverage curvature-based features, similar to those used in the PC-MSDM metric. These features capture how the shape of the point cloud is distorted.
- Color-Based Features: This is a key novelty. They extend existing 2D image color quality features to the 3D point cloud domain. They convert RGB colors to the perceptually uniform LAB2000HL color space and extract features related to lightness, chroma, and hue comparisons and contrasts.
- Optimal Feature Combination: The individual geometry and color features are then combined using logistic regression. They perform an exhaustive search to find the optimal subset of features and their corresponding weights that best predict subjective quality scores. This results in a linear model that aggregates the selected features.
Datasets
The primary dataset used for training and evaluation is the IST Rendering Point Cloud Dataset [8]. This dataset consists of 54 colored point clouds derived from 6 reference models, compressed using three different codecs (Octree-pruning, MPEG G-PCC, and MPEG V-PCC) at various quality levels. Subjective Mean Opinion Scores (MOS) were collected using a DSIS (Double Stimulus Impairment Scale) methodology from 20 subjects for each rendered point cloud. The authors specifically considered the MOS values from the session where point clouds were rendered with colors.
Key Insights and Ideas
- Color Matters Significantly: The study highlights that color information plays a crucial role in the human perception of 3D point cloud quality, often as much as, or even more than, geometric distortions.
- Complementary Nature of Features: Neither geometry nor color features alone are sufficient for accurate quality prediction. The optimal metric PCQM effectively combines both, demonstrating their complementary nature in how humans perceive visual quality.
- Lightness is Key for Color: Among the color features, those related to lightness (comparison, contrast, and structure) showed the strongest correlation with subjective scores, suggesting that variations in brightness are highly perceptible.
- Robust Correspondence is Crucial: The point-to-surface correspondence mechanism, which goes beyond simple nearest-neighbor matching, is vital for accurately capturing geometric distortions.
Key Results
- Outperforms State-of-the-Art: PCQM achieved the highest correlation with subjective MOS scores, outperforming existing metrics, including geometry-only metrics and simple distance-based measures.
- Best Model Composition: The optimized PCQM model is a linear combination of three features: Curvature structure (f3), Lightness comparison (f4), and Lightness structure (f6).
- Quantitative Performance: On the IST dataset, PCQM achieved a Pearson correlation of 0.90 and a Spearman correlation of 0.83 when evaluated over the entire dataset. When averaged over 20 test sets, it achieved a Pearson correlation of 0.873 and a Spearman correlation of 0.807.
- Code Availability: The authors have made the source code for PCQM publicly available, which is a significant contribution to the research community.
Limitations and Future Work
- Limited Dataset: While the IST dataset is comprehensive, the authors acknowledge that it's based on a specific set of reference models and compression techniques. Future work could involve evaluating PCQM on a wider variety of datasets and compression methods.
- No-Reference Metric: PCQM is a full-reference metric, meaning it requires access to the original, undistorted point cloud for comparison. Developing a no-reference metric that can predict quality without the original would be a valuable next step.
- Generalizability: While the current results are promising, further validation on diverse 3D content and distortion types would be beneficial to confirm the generalizability of the proposed metric.
- Computational Cost: While the paper focuses on accuracy, future work could also explore optimizations for real-time applications if computational efficiency becomes a bottleneck.
Summary updated