Abstract
Recent years have witnessed an explosion of user-generated content (UGC) videos shared and streamed over the Internet. Accordingly, there is a great need for accurate video quality assessment (VQA) models for UGC/consumer videos to monitor, control, and optimize this vast content. Blind quality prediction of in-the-wild videos is quite challenging, since the quality degradations of UGC content are unpredictable, complicated, and often commingled. Here we contribute to advancing the UGC-VQA problem by conducting a comprehensive evaluation of leading no-reference/blind VQA (BVQA) features and models on a fixed evaluation architecture, yielding new empirical insights on both subjective video quality studies and VQA model design. By employing a feature selection strategy on top of leading VQA model features, we are able to extract 60 of the 763 statistical features used by the leading models to create a new fusion-based BVQA model, which we dub the VIDeo quality EVALuator (VIDEVAL), that effectively balances the trade-off between VQA performance and efficiency. Our experimental results show that VIDEVAL achieves state-of-the-art performance at considerably lower computational cost than other leading models.
Introduction
Video dominates Internet traffic, much of it being UGC from platforms like YouTube, TikTok, and Facebook. Unlike "legacy" video quality assessment (VQA), which focuses on synthetic distortions (like a specific level of compression added to a perfect reference video), UGC is messy. It suffers from "authentic" distortions—blur, noise, shaky camera work, and poor lighting—often all commingled at once. Furthermore, in real-world scenarios, we don't have a "pristine" reference video to compare against. This necessitates Blind Video Quality Assessment (BVQA). Existing models often struggle to generalize across the vast diversity of UGC. This work benchmarks current models and introduces a new, more robust standard for the field.
The VIDEVAL Model
To address the limitations of existing BVQA models, we developed VIDEVAL (VIDeo quality EVALuator). The core innovation is a rigorous feature selection process. We started with a pool of 763 statistical features from top-performing models (including BRISQUE, GM-LOG, and TLVQM). Using a two-step selection strategy involving machine learning rankings (Random Forest and SVM), we distilled this down to a "best-of-the-best" set of just 60 features. This allows VIDEVAL to maintain high accuracy without the massive computational overhead of more complex models.
Datasets and Content Diversity
The study utilizes three major, large-scale UGC datasets that represent the current state of the art:
- KoNViD-1k: 1,200 public-domain videos sampled from Flickr.
- LIVE-VQC: 585 videos captured by mobile devices.
- YouTube-UGC: 1,380 video clips sampled from millions of YouTube videos.
To create a more robust benchmark, we also introduced All-Combined, a composite dataset merging all three after carefully calibrating their subjective Mean Opinion Scores (MOS).
Key Insights and Analysis
Our benchmarking revealed several critical insights into the nature of UGC quality:
- UGC is different: Traditional models designed for synthetic distortions fail to capture the complex mix of capture conditions, editing, and compression found in UGC.
- Content Space Coverage: Different datasets cover different parts of the visual attribute space. As shown in the convex hull analysis, YouTube-UGC offers a much wider variety of spatial and temporal characteristics than mobile-only datasets like LIVE-VQC.
- Resolution and Motion: Video quality is intrinsically correlated with resolution, and Natural Scene Statistics (NSS) features are implicitly "resolution-aware." Furthermore, for videos with heavy camera movement, motion-related features are crucial for accurate prediction.
Experimental Results
VIDEVAL achieves state-of-the-art or near-SOTA results across all three individual datasets and the combined benchmark. Crucially, it is significantly faster than other high-performing models like FRIQUEE, making it more practical for real-world deployment. We also found that combining VIDEVAL with deep learning features (e.g., from KonCept512) can boost performance by another ~4%, pushing the upper bound of current blind VQA capabilities.
Conclusion and Future Work
While VIDEVAL performs well on natural videos, non-natural content (animations, gaming, screen recordings) remains a major challenge due to its underrepresentation in current datasets. Future research should focus on developing models for non-natural UGC, improving the handling of temporal distortions like frame drops, and enhancing cross-dataset generalization. The implementation of VIDEVAL has been made available online to facilitate reproducible research in this growing field.
Summary updated