Abstract
Video compression is essential for modern streaming but introduces significant visual artifacts like blockiness and blurring. Compressed Video Quality Enhancement (CVQE) leverages deep learning to restore visual fidelity. However, the field lacks a unified benchmarking framework and clear taxonomy. This paper addresses these gaps by proposing a three-stage architectural framework—Feature Encoding, Propagation, and Residual Decoding—and categorizing existing methods across coding standards (AVC, HEVC, VVC). Our benchmarking reveals that while attention-based models achieve peak quality, hybrid CNN-Attention architectures offer the most efficient trade-off. We also highlight the performance gap in emerging VVC standards, providing a roadmap for future research.
Introduction
To stream high-definition content over limited bandwidth, codecs like H.264/AVC, H.265/HEVC, and H.266/VVC utilize lossy compression mechanisms. These mechanisms inevitably introduce artifacts such as ringing, blurring, and blocking. While deep learning has revolutionized the ability to "clean up" these videos, the CVQE research landscape has remained fragmented. Researchers often utilize disparate datasets and inconsistent testing conditions, making direct comparisons difficult. This paper provides a systematic classification of existing methods and a rigorous, unified benchmark to evaluate model performance across different compression standards.
A Unified Architectural Framework
A key contribution of this work is the identification of a common pipeline shared by nearly all CVQE models. This framework consists of three primary stages:
- Feature Encoding (FE): Extracting initial spatial and temporal data from the decoded frames.
- Feature Propagation (FP): Handling relationships between frames (temporal) or within a frame (spatial) to aggregate information.
- Residual Decoding (RD): Generating the final high-quality pixels by predicting the residual difference from the low-quality input.
Research Landscape and Taxonomy
The authors categorize CVQE models based on three lenses: coding standards, architectural paradigms (CNN, Attention, or Hybrid), and domain utilization (pixel-only vs. metadata-aware).
As shown in the research distribution, H.265/HEVC is currently the primary "battleground" for CVQE research. Furthermore, multi-frame (inter-frame) methods significantly outnumber single-frame (intra-frame) approaches. This trend is driven by the "inter-frame advantage," where models can "borrow" high-quality details from adjacent frames to restore a degraded current frame.
Benchmarking and Key Results
The study evaluated representative models using the JCT-VC test sequences (Classes A-E) under consistent Quantization Parameters (QP).
- Quality vs. Efficiency: Pure attention-based models like OVQE lead in quality, achieving a 1.20 dB ΔPSNR improvement on HEVC. However, they are computationally intensive (2.14 TFLOPs).
- The Hybrid Advantage: Hybrid models like CTVE emerge as the "sweet spot," achieving near-top-tier quality (0.94 dB gain) while reducing the computational load by 78% compared to OVQE.
- The VVC Frontier: Research for the newest standard, H.266/VVC, is still in its infancy. Most current VVC models are simple adaptations of HEVC architectures, resulting in lower average gains (~0.68 dB).
Conclusion and Future Directions
While CVQE has made significant strides, real-time deployment remains a major challenge. Future research should prioritize:
- VVC-Specific Architectures: Exploiting unique VVC features like its coding tree structures rather than just porting HEVC models.
- Lightweight Design: Developing models that meet the strict latency requirements of mobile and smart TV hardware.
- Compressed Domain Cues: Better utilization of "free" metadata (e.g., motion vectors) already present in the bitstream to guide enhancement.
Summary updated