Unsupervised Curriculum Domain Adaptation for No-Reference Video Quality Assessment

Pengfei Chen, Leida Li, Jinjian Wu, Weisheng Dong, Guangming Shi

Introduction

No-Reference Video Quality Assessment (NR-VQA) is critical for maintaining user experience in streaming and social media platforms. While deep learning models have shown promise, they frequently suffer from a significant generalization gap. A model trained on high-quality studio content often fails to accurately predict the quality of user-generated content (UGC) characterized by diverse, authentic distortions like camera shake or poor lighting.

Addressing this gap typically requires expensive and time-consuming collection of human subjective ratings for every new domain. Furthermore, standard NR-VQA models often predict a single Mean Opinion Score (MOS), which collapses the inherent subjectivity and disagreement among human raters. This paper introduces Unsupervised Curriculum Domain Adaptation (UCDA), a framework designed to adapt VQA models to unlabeled target domains by leveraging rating distributions and a curriculum-based learning strategy.

The UCDA Framework

The UCDA approach moves beyond scalar quality scores by predicting full rating distributions. This allows the model to capture the degree of human uncertainty and disagreement, providing a richer signal for adaptation. The framework operates in two primary stages:

1. Domain-Adaptive Quality Prediction

Initially, the model is trained on a labeled source dataset (e.g., LBVD) to predict probability distributions across quality levels (e.g., 1 to 5). To bridge the gap between source and target domains, an adversarial loss is employed. A domain classifier ($D_d$) attempts to distinguish between source and target features, while the backbone ($B$) and quality predictor ($DPH_d$) learn to generate domain-invariant features.

The UCDA architecture overview, showing the two-stage adaptation process from source to target and then within target subdomains.

2. Self-supervised Curriculum Adaptation

Not all samples in a new dataset are equally difficult for a model to evaluate. UCDA employs a curriculum learning strategy by first mastering "easy" samples before tackling "hard" ones.

  • Uncertainty Ranking: Target samples are ranked based on the uncertainty of their initial predictions. The authors use Information Theory metrics (specifically the 2-Wasserstein distance from a uniform distribution) to identify samples the model is "confident" about.
  • Subdomain Adaptation: The target domain is split into Confident and Uncertain subdomains. The confident predictions serve as pseudo-labels. A second adversarial stage ($D_s$ and $DPH_s$) then aligns the features of the uncertain samples with those of the confident ones, effectively "rectifying" the model's performance on difficult cases.

Experimental Evaluation

The authors evaluated UCDA using the LBVD database as the source and five diverse target databases, including both artificial (LIVE, CSIQ) and authentic (KoNViD-1k, LIVE-VQC) distortions.

Impact of the Curriculum

A key hyperparameter in UCDA is $\eta$, which controls the ratio of samples assigned to the confident subdomain. As shown in the performance analysis, there is a clear "sweet spot" for this parameter across different datasets, typically around $\eta=0.6$.

Performance (PLCC) on various target datasets as a function of the curriculum parameter η, showing optimal adaptation around η=0.6.

The results indicate that UCDA consistently outperforms existing unsupervised domain adaptation methods like TCoN and TA³N. On the large-scale KoNViD-1k dataset, UCDA achieved a PLCC of 0.7909, rivaling some supervised models trained directly on other VQA datasets.

Conclusion

UCDA presents a robust solution for the generalization challenges in NR-VQA. By treating quality assessment as a distribution prediction task and implementing a curriculum that bridges confident and uncertain samples, the model achieves state-of-the-art unsupervised performance. With an execution speed of over 67 FPS on 720p videos, the framework is well-suited for real-time industrial applications. Future work may involve integrating visual attention mechanisms to better handle complex scenes that currently contribute to model uncertainty.

Summary updated

Abstract

During the last years, convolutional neural networks (C-NNs) have triumphed over video quality assessment (VQA) tasks. However, CNN-based approaches heavily rely on annotated data which are typically not available in VQA, leading to the difficulty of model generalization. Recent advances in domain adaptation technique makes it possible to adapt models trained on source data to unlabeled target data. However, due to the distortion diversity and content variation of the collected videos, the intrinsic subjectivity of VQA tasks hampers the adaptation performance. In this work, we propose a curriculum-style unsupervised domain adaptation to handle the cross-domain no-reference VQA problem. The proposed approach could be divided into two stages. In the first stage, we conduct an adaptation between source and target domains to predict the rating distribution for target samples, which can better reveal the subjective nature of VQA. From this adaptation, we split the data in target domain into confident and uncertain subdomains using the proposed uncertainty-based ranking function, through measuring their prediction confidences. In the second stage, by regarding samples in confident subdomain as the easy tasks in the curriculum, a fine-level adaptation is conducted between two subdomain-s to fine-tune the prediction model. Extensive experimental results on benchmark datasets highlight the superiority of the proposed method over the competing methods in both accuracy and speed. The source code is released at https://github.com/cpf0079/UCDA.
BibTeX
@inproceedings{chen2021unsupervised,
  author = {Pengfei Chen and Leida Li and Jinjian Wu and Weisheng Dong and Guangming Shi},
  title = {{Unsupervised Curriculum Domain Adaptation for No-Reference Video Quality Assessment}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00513},
}
Figure