Abstract
Predicting human perception of video quality under compression and network instability is a complex challenge. This study proposes a No-Reference (NR) Video Quality Assessment (VQA) framework that leverages 3D Convolutional Neural Networks (3D-CNNs) to extract spatial and temporal features directly from raw video data. By integrating network parameters into the preprocessing stage, the model achieves significant accuracy in classifying perceived quality levels for user-generated content.
Introduction
Assessing video quality is inherently subjective. While traditional metrics like Peak Signal-to-Noise Ratio (PSNR) are computationally efficient, they often fail to correlate with human visual perception, particularly for modern standards like HEVC (H.265). In these cases, compression artifacts are unpredictable, and mobile streaming introduces further degradation through network congestion, latency, and packet loss.
Most existing VQA methods are "Full-Reference," requiring the original uncompressed video for comparison. However, in real-world streaming scenarios, the receiver only has access to the degraded signal. This paper addresses the "No-Reference" (NR) challenge: predicting quality using only the degraded video and known network conditions.
Methodology
The core of the proposed system is a 3D Convolutional Neural Network (3D-CNN) designed to capture both spatial and temporal features. Unlike 2D-CNNs, 3D kernels extract features across consecutive frames, which is essential for identifying motion-related artifacts.
Key technical components include:
- Network-Aware Input: The model estimates throughput ($R$) based on Round-Trip Time (RTT) and packet loss. A derived "delay" ($\delta$) is manually appended to the start of the raw pixel data, embedding network performance directly into the visual signal.
- Patch-Based Learning: To manage the memory constraints of high-resolution video ($1920 \times 1080$), the input is divided into $16 \times 16 \times 16$ cubic patches.
- Aggregation Strategies: Patch-level predictions are aggregated into a final video-level score using either Majority Voting or a Pre-trained 1D-CNN.
Experimental Evaluation
Due to the lack of suitable existing datasets, the authors developed a custom VQA dataset using three User-Generated Content (UGC) sequences from the Edinburgh Festival.
The videos were encoded using the Turing HEVC codec at five different Quantisation Parameters (QP 18 to 40) and subjected to four simulated network conditions (RTT: 5ms–50ms; loss: 0.1%–3%). Ground truth was established via Mean Opinion Scores (MOS) from eight human subjects, which were discretized into quality classes.
Results and Discussion
The end-to-end learning approach outperformed traditional methods relying on hand-crafted features. Key findings include:
- Classification Accuracy: The system achieved 80% accuracy in classifying videos into three quality categories (Low, Mid, High). Individual patches were classified with 74% to 77% accuracy.
- Aggregation Performance: Interestingly, the simpler Majority Voting strategy slightly outperformed the 1D-CNN approach in the 3-class scenario.
- Optimization: The Adagrad optimizer proved more stable than standard SGD, likely due to its ability to handle sparse gradients in patch-based data.
- Perceptual Sensitivity: The model was most accurate at the extremes of the quality spectrum ("Excellent" or "Bad"), mirroring human perceptual tendencies.
Conclusion
This research demonstrates the efficacy of 3D-CNNs for NR-VQA, particularly when augmented with network-layer information. Future work will focus on expanding the dataset diversity, transitioning from classification to continuous regression scores, and incorporating chrominance channels to account for color distortions.
Summary updated