TL;DR
This paper introduces LS-PCQA, the largest-ever dataset for point cloud quality assessment, featuring over 22,000 distorted samples across 31 impairment types. They also propose ResSCNN, a novel no-reference metric based on sparse convolutional neural networks, which achieves state-of-the-art performance, even outperforming some full-reference metrics.
What Problem Does This Paper Address?
Assessing the quality of 3D point clouds is crucial for applications like augmented reality and autonomous driving. While full-reference (FR) methods, which require an original "ground truth" point cloud, have seen progress, obtaining these references is often impractical or impossible in real-world scenarios. This limitation highlights the need for no-reference (NR) metrics that can predict quality from distorted point clouds alone. However, the development of effective NR metrics has been hampered by the lack of large-scale, diverse datasets for training and evaluation. Existing datasets are too small and lack the variety of distortions needed to train robust NR models.
Methods
The authors tackle this problem in three main steps:
- Dataset Construction (LS-PCQA): They create a large-scale dataset by collecting 104 high-quality reference point clouds and generating over 22,000 distorted samples. These distortions cover 31 types, including geometric (e.g., local missing, down-sampling) and photometric (e.g., Gaussian noise, contrast distortion) impairments, each at 7 different levels.
- Pseudo-MOS Annotation: Due to the cost and time of subjective experiments, they employ a "pseudo-MOS" approach. They first conduct a limited subjective experiment on a subset of the data to identify the best-performing FR metrics for each distortion type. Then, they use these selected FR metrics and a nonlinear regression model (specifically, Logistic-5) to automatically generate quality scores (pseudo-MOS) for the rest of the dataset.
- ResSCNN Metric: They propose ResSCNN, a novel NR metric based on a sparse convolutional neural network (CNN). Unlike traditional CNNs that can struggle with the irregular structure of point clouds, sparse CNNs efficiently process the scattered data. ResSCNN employs a hierarchical feature extraction module using sparse convolutions to capture both local and global information without dimensionality reduction, followed by pooling, concatenation, and fully connected layers for quality prediction.
Datasets
- LS-PCQA (Newly Introduced): This is the primary dataset introduced in the paper. It contains 104 reference point clouds and 22,568 distorted samples, generated with 31 types of impairments at 7 distortion levels.
- SJTU-PCQA: An existing dataset used for evaluation.
- WPC2.0: Another existing dataset used for evaluation.
Key Insights and Ideas
- The critical need for large-scale, diverse datasets: The paper strongly emphasizes that the lack of such datasets is the primary bottleneck for advancing NR-PCQA. Their LS-PCQA dataset aims to fill this gap.
- Pseudo-MOS as a viable annotation strategy: By leveraging the strengths of existing FR metrics and a robust nonlinear mapping, they demonstrate that pseudo-MOS can accurately approximate subjective quality scores, enabling the creation of large datasets.
- Sparse CNNs are well-suited for point cloud quality assessment: The irregular and sparse nature of point clouds makes them a natural fit for sparse convolutional operations, which avoid the computational overhead and information loss associated with dense convolutions or dimensionality reduction techniques.
- Hierarchical features are crucial: Combining features from different network depths (shallow for detailed distortions, deep for semantic ones) leads to more robust quality prediction.
Key Results
- State-of-the-art performance: ResSCNN achieves state-of-the-art (SOTA) performance among existing NR-PCQA metrics on the LS-PCQA, SJTU-PCQA, and WPC2.0 datasets.
- Competitive with FR metrics: Remarkably, ResSCNN often outperforms several FR metrics, demonstrating the effectiveness of their NR approach. For instance, on LS-PCQA, ResSCNN achieved a PLCC of 0.60 and SROCC of 0.62, comparable to MPED's 0.70 and 0.67.
- Dataset impact: Pre-training on the LS-PCQA dataset significantly improves the generalization capability of NR metrics on other datasets, showing a substantial increase in PLCC and SROCC (up to 40% and 33% respectively).
- No dimensionality reduction needed: Experiments show that dimensionality reduction techniques (like down-sampling) negatively impact NR-PCQA performance, reinforcing the value of using complete point clouds.
Limitations and Future Work
- Limitations of Pseudo-MOS: While effective, pseudo-MOS is still an approximation of subjective scores. The paper acknowledges that the accuracy of pseudo-MOS can vary across distortion types.
- Color Distortions: The paper notes that NR-PCQA metrics, including their own, tend to perform poorly on some color distortions, suggesting a need for better feature extraction for these types of impairments.
- Potential for Improvement: The authors suggest that both FR and NR metrics for PCQA have room for further improvement, especially given the increasing complexity of point cloud data and distortions. Future work could focus on developing more sophisticated feature representations and exploring more advanced network architectures.
Summary updated