Abstract
Because of the increasing ease of video capture, many millions of consumers create and upload large volumes of User-Generated-Content (UGC) videos to social and streaming media sites over the Internet. UGC videos are commonly captured by naive users having limited skills and imperfect techniques, and tend to be afflicted by mixtures of highly diverse in-capture distortions. These UGC videos are then often uploaded for sharing onto cloud servers, where they are further compressed for storage and transmission. Our paper tackles the highly practical problem of predicting the quality of compressed videos (perhaps during the process of compression, to help guide it), with only (possibly severely) distorted UGC videos as references. To address this problem, we have developed a novel Video Quality Assessment (VQA) framework that we call 1stepVQA. 1stepVQA overcomes limitations of Full-Reference, Reduced-Reference and No-Reference VQA models by exploiting the statistical regularities of both natural videos and distorted videos. We also describe a new dedicated video database, which was created by applying a realistic VMAF-Guided perceptual rate distortion optimization (RDO) criterion to create realistically compressed versions of UGC source videos, which typically have pre-existing distortions. We show that 1stepVQA is able to more accurately predict the quality of compressed videos, given imperfect reference videos, and outperforms other VQA models in this scenario.
Introduction
Traditional Video Quality Assessment (VQA) models typically operate under the assumption of a "pristine" reference—a high-quality, distortion-free version of the video used as a benchmark to measure compression loss. However, in the modern landscape of social media and streaming, this assumption is frequently violated. Most User-Generated Content (UGC) is captured by amateur users on mobile devices, resulting in videos that suffer from "in-capture" distortions such as camera shake, poor lighting, and sensor noise before they are ever uploaded and compressed by cloud servers.
When standard Full-Reference (FR) models are applied to these videos, they often fail because they compare a distorted compressed video against an already distorted reference, leading to inaccurate quality predictions. Conversely, No-Reference (NR) models struggle to disentangle the complex, overlapping distortions inherent in UGC. This paper introduces 1stepVQA, a framework specifically designed for the "distorted-then-compressed" scenario, leveraging the statistical regularities of both natural and distorted videos to provide more accurate quality predictions.
The 1stepVQA Framework
Unlike previous "two-step" methods that attempt to predict source quality and compression loss independently, 1stepVQA analyzes features from both the source and compressed videos simultaneously. This integrated approach allows the model to learn the non-linear interactions between native UGC distortions and subsequent compression artifacts.
Technical Components
- Natural Scene Statistics (NSS): The model is predicated on the observation that high-quality natural videos follow specific statistical regularities, which are predictably disrupted by distortions.
- Natural Frame Statistics (NFS): Spatial distortions (e.g., blur or noise) are captured within individual frames using Mean Subtracted Contrast Normalized (MSCN) coefficients.
- Natural Video Statistics (NVS) via Displaced Differences: To capture temporal information efficiently, the model utilizes displaced frame differences. By shifting one frame diagonally before subtraction, the model captures motion-related information and temporal regularities without the high computational cost of full motion estimation.
- Support Vector Regression (SVR): A total of 40 features, extracted at two spatial scales, are fed into an SVR to map statistical deviations to a final quality score.
The LIVE Wild Compressed Video Quality Database
To evaluate the model, the authors developed the LIVE Wild Compressed Video Quality Database, as existing datasets did not adequately represent the distorted-then-compressed pipeline.
- Source Material: 55 unique UGC videos were selected from the LIVE VQC database, representing a wide array of authentic, real-world distortions.
- Compression Strategy: Each source was compressed into four versions using H.264. To ensure perceptual distinctness, the authors employed "convex hull" curves based on VMAF scores to select optimal bitrate and resolution combinations.
- Subjective Ground Truth: Mean Opinion Scores (MOS) were collected from 40 human subjects in a controlled laboratory environment to provide reliable quality labels.
Experimental Results
1stepVQA demonstrated superior performance across several key metrics compared to industry-standard VQA models:
- Accuracy: The model achieved a Spearman Rank Ordered Correlation Coefficient (SROCC) of 0.8839, significantly outperforming standard FR models like PSNR (0.5084) and NR models like BRISQUE (0.7877).
- Robustness: It surpassed the previous state-of-the-art "2stepQA" model (0.8493), validating the efficacy of the integrated one-step approach.
- Efficiency: Timing tests indicate that 1stepVQA is computationally efficient and significantly faster than complex temporal models like TLVQM or V-BLIINDS, making it suitable for real-time cloud processing pipelines.
Conclusion
1stepVQA addresses a critical gap in video quality assessment by explicitly modeling the relationship between pre-existing capture distortions and compression artifacts. By leveraging displaced frame differences and an integrated SVR-based framework, it provides a robust and efficient solution for monitoring UGC quality. Future work may explore the integration of deep learning architectures and phase-based features to further enhance predictive accuracy as larger UGC datasets become available.
Summary updated