Compressed Video Quality Enhancement: Classifying and Benchmarking over Standards

Xiem HoangVan, Dang Bui Dinh, Sang NguyenQuang, Wen-Hsiao PengarXiv 2025 · DOI · arXiv

Abstract

Video compression is essential for modern streaming but introduces significant visual artifacts like blockiness and blurring. Compressed Video Quality Enhancement (CVQE) leverages deep learning to restore visual fidelity. However, the field lacks a unified benchmarking framework and clear taxonomy. This paper addresses these gaps by proposing a three-stage architectural framework—Feature Encoding, Propagation, and Residual Decoding—and categorizing existing methods across coding standards (AVC, HEVC, VVC). Our benchmarking reveals that while attention-based models achieve peak quality, hybrid CNN-Attention architectures offer the most efficient trade-off. We also highlight the performance gap in emerging VVC standards, providing a roadmap for future research.

Introduction

To stream high-definition content over limited bandwidth, codecs like H.264/AVC, H.265/HEVC, and H.266/VVC utilize lossy compression mechanisms. These mechanisms inevitably introduce artifacts such as ringing, blurring, and blocking. While deep learning has revolutionized the ability to "clean up" these videos, the CVQE research landscape has remained fragmented. Researchers often utilize disparate datasets and inconsistent testing conditions, making direct comparisons difficult. This paper provides a systematic classification of existing methods and a rigorous, unified benchmark to evaluate model performance across different compression standards.

A Unified Architectural Framework

A key contribution of this work is the identification of a common pipeline shared by nearly all CVQE models. This framework consists of three primary stages:

  1. Feature Encoding (FE): Extracting initial spatial and temporal data from the decoded frames.
  2. Feature Propagation (FP): Handling relationships between frames (temporal) or within a frame (spatial) to aggregate information.
  3. Residual Decoding (RD): Generating the final high-quality pixels by predicting the residual difference from the low-quality input.
The three-stage CVQE pipeline consisting of Feature Encoding, Propagation, and Residual Decoding.

Research Landscape and Taxonomy

The authors categorize CVQE models based on three lenses: coding standards, architectural paradigms (CNN, Attention, or Hybrid), and domain utilization (pixel-only vs. metadata-aware).

Distribution of CVQE research across different video coding standards and frame processing types.

As shown in the research distribution, H.265/HEVC is currently the primary "battleground" for CVQE research. Furthermore, multi-frame (inter-frame) methods significantly outnumber single-frame (intra-frame) approaches. This trend is driven by the "inter-frame advantage," where models can "borrow" high-quality details from adjacent frames to restore a degraded current frame.

Benchmarking and Key Results

The study evaluated representative models using the JCT-VC test sequences (Classes A-E) under consistent Quantization Parameters (QP).

  • Quality vs. Efficiency: Pure attention-based models like OVQE lead in quality, achieving a 1.20 dB ΔPSNR improvement on HEVC. However, they are computationally intensive (2.14 TFLOPs).
  • The Hybrid Advantage: Hybrid models like CTVE emerge as the "sweet spot," achieving near-top-tier quality (0.94 dB gain) while reducing the computational load by 78% compared to OVQE.
  • The VVC Frontier: Research for the newest standard, H.266/VVC, is still in its infancy. Most current VVC models are simple adaptations of HEVC architectures, resulting in lower average gains (~0.68 dB).

Conclusion and Future Directions

While CVQE has made significant strides, real-time deployment remains a major challenge. Future research should prioritize:

  1. VVC-Specific Architectures: Exploiting unique VVC features like its coding tree structures rather than just porting HEVC models.
  2. Lightweight Design: Developing models that meet the strict latency requirements of mobile and smart TV hardware.
  3. Compressed Domain Cues: Better utilization of "free" metadata (e.g., motion vectors) already present in the bitstream to guide enhancement.

Summary updated

Abstract

Compressed video quality enhancement (CVQE) is crucial for improving user experience with lossy video codecs like H.264/AVC, H.265/HEVC, and H.266/VVC. While deep learning based CVQE has driven significant progress, existing surveys still suffer from limitations: lack of systematic classification linking methods to specific standards and artifacts, insufficient comparative analysis of architectural paradigms across coding types, and underdeveloped benchmarking practices. To address these gaps, this paper presents three key contributions. First, it introduces a novel taxonomy classifying CVQE methods across architectural paradigms, coding standards, and compressed-domain feature utilization. Second, it proposes a unified benchmarking framework integrating modern compression protocols and standard test sequences for fair multi-criteria evaluation. Third, it provides a systematic analysis of the critical trade-offs between reconstruction performance and computational complexity observed in state-of-the-art methods and highlighting promising directions for future research. This comprehensive review aims to establish a foundation for consistent assessment and informed model selection in CVQE research and deployment.
BibTeX
@misc{hoangvan2025compressed,
  author = {Xiem HoangVan and Dang Bui Dinh and Sang NguyenQuang and Wen-Hsiao Peng},
  title = {{Compressed Video Quality Enhancement: Classifying and Benchmarking over Standards}},
  year = {2025},
  eprint = {2509.10407},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.10407},
}
Figure