Point Cloud Quality Assessment: Dataset Construction and Learning-based No-reference Metric

Yipeng Liu, Qi Yang, Yiling Xu, Le Yang

TL;DR

This paper introduces LS-PCQA, the largest-ever dataset for point cloud quality assessment, featuring over 22,000 distorted samples across 31 impairment types. They also propose ResSCNN, a novel no-reference metric based on sparse convolutional neural networks, which achieves state-of-the-art performance, even outperforming some full-reference metrics.

What Problem Does This Paper Address?

Assessing the quality of 3D point clouds is crucial for applications like augmented reality and autonomous driving. While full-reference (FR) methods, which require an original "ground truth" point cloud, have seen progress, obtaining these references is often impractical or impossible in real-world scenarios. This limitation highlights the need for no-reference (NR) metrics that can predict quality from distorted point clouds alone. However, the development of effective NR metrics has been hampered by the lack of large-scale, diverse datasets for training and evaluation. Existing datasets are too small and lack the variety of distortions needed to train robust NR models.

Methods

The authors tackle this problem in three main steps:

  1. Dataset Construction (LS-PCQA): They create a large-scale dataset by collecting 104 high-quality reference point clouds and generating over 22,000 distorted samples. These distortions cover 31 types, including geometric (e.g., local missing, down-sampling) and photometric (e.g., Gaussian noise, contrast distortion) impairments, each at 7 different levels.
  2. Pseudo-MOS Annotation: Due to the cost and time of subjective experiments, they employ a "pseudo-MOS" approach. They first conduct a limited subjective experiment on a subset of the data to identify the best-performing FR metrics for each distortion type. Then, they use these selected FR metrics and a nonlinear regression model (specifically, Logistic-5) to automatically generate quality scores (pseudo-MOS) for the rest of the dataset.
  3. ResSCNN Metric: They propose ResSCNN, a novel NR metric based on a sparse convolutional neural network (CNN). Unlike traditional CNNs that can struggle with the irregular structure of point clouds, sparse CNNs efficiently process the scattered data. ResSCNN employs a hierarchical feature extraction module using sparse convolutions to capture both local and global information without dimensionality reduction, followed by pooling, concatenation, and fully connected layers for quality prediction.

Datasets

  • LS-PCQA (Newly Introduced): This is the primary dataset introduced in the paper. It contains 104 reference point clouds and 22,568 distorted samples, generated with 31 types of impairments at 7 distortion levels.
  • SJTU-PCQA: An existing dataset used for evaluation.
  • WPC2.0: Another existing dataset used for evaluation.

Key Insights and Ideas

  • The critical need for large-scale, diverse datasets: The paper strongly emphasizes that the lack of such datasets is the primary bottleneck for advancing NR-PCQA. Their LS-PCQA dataset aims to fill this gap.
  • Pseudo-MOS as a viable annotation strategy: By leveraging the strengths of existing FR metrics and a robust nonlinear mapping, they demonstrate that pseudo-MOS can accurately approximate subjective quality scores, enabling the creation of large datasets.
  • Sparse CNNs are well-suited for point cloud quality assessment: The irregular and sparse nature of point clouds makes them a natural fit for sparse convolutional operations, which avoid the computational overhead and information loss associated with dense convolutions or dimensionality reduction techniques.
  • Hierarchical features are crucial: Combining features from different network depths (shallow for detailed distortions, deep for semantic ones) leads to more robust quality prediction.

Key Results

  • State-of-the-art performance: ResSCNN achieves state-of-the-art (SOTA) performance among existing NR-PCQA metrics on the LS-PCQA, SJTU-PCQA, and WPC2.0 datasets.
  • Competitive with FR metrics: Remarkably, ResSCNN often outperforms several FR metrics, demonstrating the effectiveness of their NR approach. For instance, on LS-PCQA, ResSCNN achieved a PLCC of 0.60 and SROCC of 0.62, comparable to MPED's 0.70 and 0.67.
  • Dataset impact: Pre-training on the LS-PCQA dataset significantly improves the generalization capability of NR metrics on other datasets, showing a substantial increase in PLCC and SROCC (up to 40% and 33% respectively).
  • No dimensionality reduction needed: Experiments show that dimensionality reduction techniques (like down-sampling) negatively impact NR-PCQA performance, reinforcing the value of using complete point clouds.

Limitations and Future Work

  • Limitations of Pseudo-MOS: While effective, pseudo-MOS is still an approximation of subjective scores. The paper acknowledges that the accuracy of pseudo-MOS can vary across distortion types.
  • Color Distortions: The paper notes that NR-PCQA metrics, including their own, tend to perform poorly on some color distortions, suggesting a need for better feature extraction for these types of impairments.
  • Potential for Improvement: The authors suggest that both FR and NR metrics for PCQA have room for further improvement, especially given the increasing complexity of point cloud data and distortions. Future work could focus on developing more sophisticated feature representations and exploring more advanced network architectures.

Summary updated

Abstract

Full-reference (FR) point cloud quality assessment (PCQA) has achieved impressive progress in recent years. However, in many cases, obtaining the reference point clouds is difficult, so no-reference (NR) metrics have become a research hotspot. Few researches about NR-PCQA are carried out due to the lack of a large-scale PCQA dataset. In this article, we first build a large-scale PCQA dataset named LS-PCQA, which includes 104 reference point clouds and more than 22,000 distorted samples. In the dataset, each reference point cloud is augmented with 31 types of impairments (e.g., Gaussian noise, contrast distortion, local missing, and compression loss) at 7 distortion levels. Besides, each distorted point cloud is assigned with a pseudo-quality score as its substitute of Mean Opinion Score. Inspired by the hierarchical perception system and considering the intrinsic attributes of point clouds, we propose a NR metric ResSCNN based on sparse convolutional neural network (CNN) to accurately estimate the subjective quality of point clouds. We conduct several experiments to evaluate the performance of the proposed NR metric. The results demonstrate that ResSCNN exhibits the state-of-the-art performance among all the existing NR-PCQA metrics and even outperforms some FR metrics. The dataset presented in this work will be made publicly accessible at https://smt.sjtu.edu.cn. The source code for the proposed ResSCNN can be found at https://github.com/lyp22/ResSCNN.
BibTeX
@article{liu2023point,
  author = {Yipeng Liu and Qi Yang and Yiling Xu and Le Yang},
  title = {{Point Cloud Quality Assessment: Dataset Construction and Learning-based No-reference Metric}},
  journal = {ACM Trans. Multim. Comput. Commun. Appl.},
  year = {2023},
  url = {https://openreview.net/forum?id=ukDzp9a9Kj},
}
Figure