Introduction
Point clouds are essential for 3D applications such as autonomous driving and virtual reality. However, compression and transmission often introduce visual distortions, necessitating robust Point Cloud Quality Assessment (PCQA). Traditional No-Reference (NR) methods typically treat quality assessment as a deterministic mapping from a 3D object to a single Mean Opinion Score (MOS).
This approach overlooks the stochastic nature of human perception; different observers often provide varying scores for the same stimulus due to subjective biases, mood, or context. By targeting a single average value, deterministic models may become overconfident and fail to capture the underlying distribution of human judgment. To address this, researchers have developed a probabilistic framework that accounts for the inherent uncertainty in human perception.
Methods
The proposed Stochasticity-aware NR-PCQA framework is built upon a Conditional Variational Autoencoder (CVAE). This architecture allows the model to simulate the diverse range of scores different humans might assign to a 3D object.
Framework Components
- Input Processing: To manage computational costs, the point cloud is projected into multiple 2D RGB-D (color + depth) images from random viewpoints. This random selection mimics interactive human viewing and serves as a form of data augmentation.
- CVAE Architecture:
- Prior Module ($P_\theta(z|I)$): Predicts the distribution of potential quality judgments based solely on the point cloud projections.
- Posterior Module ($Q_\phi(z|I, M)$): Utilized during training, this module incorporates the ground-truth MOS ($M$) to refine the latent distribution.
- Quality Rating Generator ($P_\omega(M|I, z)$): This component takes a sampled "stochastic feature" from the distribution along with the image projections to produce a quality rating.
- Training and Inference: The model employs KL Annealing to prevent posterior collapse and uses a Gaussian Stochastic Neural Network (GSNN) loss. During inference, the model samples from the learned distribution multiple times to generate several "subjective" ratings, which are then averaged to produce the final quality score—effectively mimicking the calculation of a real MOS from a human subject group.
Key Insights
- Embracing Uncertainty: Modeling the uncertainty of human subjects improves the accuracy of the deterministic mean score prediction.
- The Role of Depth: Ablation studies confirmed that depth information is crucial; removing it significantly degrades performance, proving that geometry is as vital as color for 3D quality assessment.
- Late Averaging: The researchers found that generating multiple ratings and averaging them at the end of the pipeline ("Late Average") outperformed averaging features early in the process.
Key Results
The proposed method achieved state-of-the-art (SOTA) performance across three major benchmark datasets: SJTU-PCQA, WPC (Waterloo Point Cloud), and WPC2.0.
As shown in the results table, the model consistently outperforms existing NR-PCQA methods. On the challenging WPC2.0 dataset, it achieved an SRCC of 0.8742, surpassing the previous best method (MM-PCQA) by over 7%. Furthermore, the model demonstrated exceptional cross-dataset generalization; when trained on WPC and tested on SJTU, it achieved an SRCC of 0.9133, significantly higher than the next best competitor (0.7693). Remarkably, this No-Reference model even outperformed several Full-Reference methods that have access to the original undistorted point clouds.
Limitations and Future Work
While the model is highly effective, the multiple sampling required during inference introduces some computational overhead compared to one-shot deterministic models. Additionally, while the framework handles the absence of raw individual scores well, access to raw subjective data during training could further enhance performance. Future research may explore applying this stochastic modeling approach to other subjective assessment tasks, such as image or video quality evaluation.
Summary updated