Introduction
Video compression is a fundamental challenge in digital media, balancing the need for reduced file sizes with the preservation of visual fidelity. The High Efficiency Video Coding (HEVC) standard relies on two pivotal stages: intra prediction, which estimates pixel blocks from spatial neighbors, and reconstruction filtering, which mitigates compression artifacts like "blocking." While neural networks have been applied to these stages individually, existing solutions often lack cohesive integration and fail to capture the multi-scale features inherent in complex video data.
This research introduces a "dual-threat" framework that integrates deep learning into both intra prediction and reconstruction filtering, achieving significant bitrate reductions while maintaining high visual quality.
The Dual-Learning Framework
The proposed architecture modifies the standard HEVC pipeline through two primary innovations:
1. FC-based Intra Prediction
To enhance spatial prediction, the authors introduce a 36th prediction mode powered by a fully-connected (FC) neural network. This mode operates at the Prediction Unit (PU) level. A single signaling bit is added to the bitstream to inform the decoder whether to use this learned mode or a traditional directional prediction mode.
2. Inception-Dense Reconstruction Filtering
The core of the reconstruction stage is a novel convolutional neural network (CNN) designed for block-level filtering. Unlike traditional frame-level filters, this approach allows cleaned-up pixels to serve as more accurate references for subsequent block predictions within the encoding loop.
As shown in the figure above, the architecture utilizes 12 "Improved Inception" blocks. These blocks employ multiple parallel branches with varying kernel sizes ($1 \times 1$, $1 \times 3$, $3 \times 1$, and $3 \times 3$) to extract features across different scales simultaneously. This multi-scale approach is significantly more effective at capturing complex textures than the fixed-kernel layers found in baseline models like VR-CNN.
Experimental Results
The model was trained on the high-resolution DIV2K dataset and evaluated using standard HEVC Test Sequences.
Performance Gains
The dual-learning approach demonstrated a "superposition effect," where the combination of prediction and filtering outperformed the sum of their individual contributions. Specifically, the integrated model achieved an average BD-rate saving of 10.24% for All-Intra (AI) configurations compared to the HEVC reference software (HM-16.20). This represents a substantial improvement over established benchmarks like AR-CNN and VR-CNN.
Visual Quality
Beyond numerical metrics, the framework provides noticeable improvements in subjective visual quality. The Inception-dense blocks are particularly adept at reducing "contouring" and "blocking" artifacts.
As illustrated in the facial region above, the proposed method preserves natural textures and edges more effectively than standard HEVC, resulting in a more pleasing viewing experience.
Conclusion and Future Work
While the proposed model delivers state-of-the-art compression performance with a relatively lean parameter count (approx. 475,000), it introduces significant computational overhead. Decoding times can increase by over 100,000%, necessitating high-end GPUs for current implementations.
Future research will focus on developing "lighter" versions of these Inception-dense blocks to reduce the computational burden, aiming to bring these significant bitrate savings to consumer-grade hardware and real-time applications.
Summary updated