A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC

Tianyi Li, Mai Xu, Ren Yang, Xiaoming Tao

Abstract

High efficiency video coding (HEVC) has brought outperforming efficiency for video compression. To reduce the compression artifacts of HEVC, we propose a DenseNet based approach as the in-loop filter of HEVC, which leverages multiple adjacent frames to enhance the quality of each encoded frame. Specifically, the higher-quality frames are found by a reference frame selector (RFS). Then, a deep neural network for multi-frame in-loop filter (named MIF-Net) is developed to enhance the quality of each encoded frame by utilizing the spatial information of this frame and the temporal information of its neighboring higher-quality frames. The MIF-Net is built on the recently developed DenseNet, benefiting from the improved generalization capacity and computational efficiency. Finally, experimental results verify the effectiveness of our multi-frame in-loop filter, outperforming the HM baseline and other state-of-the-art approaches.

Introduction

Video compression is a fundamental challenge in balancing file size with visual fidelity. The High Efficiency Video Coding (HEVC) standard employs block-wise prediction and quantization which, while effective, frequently introduce artifacts such as blocking, blurring, and ringing. Standard HEVC attempts to mitigate these issues using in-loop filters like the Deblocking Filter (DBF) and Sample Adaptive Offset (SAO). However, these traditional filters typically operate on a single frame, neglecting the temporal quality fluctuations inherent in compressed video streams.

This paper proposes a novel approach centered on "temporal quality mining." By identifying high-quality neighboring frames and using them as "quality donors," the proposed system can effectively repair artifacts in lower-quality frames. This is achieved through a deep learning framework that integrates spatial and temporal information directly into the HEVC encoding loop.

Methods

The proposed system consists of two primary components integrated into the HEVC pipeline: the Reference Frame Selector (RFS) and the Multi-frame In-loop Filter Network (MIF-Net).

Reference Frame Selector (RFS)

Not all neighboring frames are equally beneficial for enhancement. The RFS utilizes a small neural network, RFS-Net, to evaluate the preceding 16 frames. It selects the top two reference frames ($M=2$) that exhibit both superior quality (measured by PSNR) and high content correlation with the current frame.

MIF-Net Architecture

The MIF-Net is the core deep learning model designed for quality enhancement. It incorporates several key innovations:

  • Modified Motion Compensation: To account for movement between frames, the network uses a modified Spatial Transformer Motion Compensation (STMC) to align reference frames with the target frame.
  • Block-adaptive Convolutional Layer: Since artifacts are often concentrated at block boundaries, this layer utilizes the encoder's own partition maps (CU and TU maps) as input. This guides the network to focus its restoration efforts on specific block boundaries.
  • Dense Units: Drawing inspiration from DenseNet, the architecture employs dense connections to promote feature reuse, enhancing both generalization and computational efficiency.
Overall framework of the proposed multi-frame in-loop filter in HEVC.

As illustrated in the architecture diagram below, the MIF-Net processes the unfiltered reconstructed frame (URF) alongside the compensated reference frames and partition metadata to produce an enhanced output. In cases where suitable reference frames are not found, the system falls back to a single-frame IF-Net.

Architecture of the MIF-Net and IF-Net, detailing the flow from input frames and metadata to the final enhanced output.

Training and Datasets

To train the model, the authors developed the HIF database, a large-scale benchmark containing 111 raw video sequences. This diverse dataset ensures the model can generalize across various motion types and content categories. Training was conducted in two stages using intermediate supervision to first optimize motion alignment before focusing on final image quality.

Key Results

The proposed MIF approach was rigorously tested against the standard HEVC (HM 16.5) and other state-of-the-art deep learning filters. The experimental results demonstrate significant improvements:

  • Bit-rate Savings: The method achieved an average BD-BR saving of 11.62%, allowing for equivalent visual quality at a substantially lower data rate.
  • Objective Quality: There was an average BD-PSNR improvement of 0.39 dB.
  • Comparative Performance: The MIF-Net outperformed the previous state-of-the-art filter, RHCNN, by 2.40% in bit-rate savings.
  • Subjective Quality: Visual analysis confirmed sharper edges on moving objects and significantly reduced background noise compared to standard HEVC filters.

Conclusion

This research demonstrates the power of leveraging temporal information for in-loop filtering in video compression. By combining a reference frame selection mechanism with a DenseNet-based enhancement network, the proposed MIF-Net significantly reduces compression artifacts. While the current model offers impressive quality gains, future research will focus on model compression and optimization to facilitate real-time implementation in consumer hardware, where the computational overhead of deep learning remains a primary constraint.

Summary updated

Abstract

High efficiency video coding (HEVC) has brought outperforming efficiency for video compression. To reduce the compression artifacts of HEVC, we propose a DenseNet based approach as the in-loop filter of HEVC, which leverages multiple adjacent frames to enhance the quality of each encoded frame. Specifically, the higher-quality frames are found by a reference frame selector (RFS). Then, a deep neural network for multi-frame in-loop filter (named MIF-Net) is developed to enhance the quality of each encoded frame by utilizing the spatial information of this frame and the temporal information of its neighboring higher-quality frames. The MIF-Net is built on the recently developed DenseNet, benefiting from the improved generalization capacity and computational efficiency. Finally, experimental results verify the effectiveness of our multi-frame in-loop filter, outperforming the HM baseline and other state-of-the-art approaches.
BibTeX
@inproceedings{li2019densenet,
  author = {Tianyi Li and Mai Xu and Ren Yang and Xiaoming Tao},
  title = {{A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC}},
  booktitle = {DCC},
  year = {2019},
  url = {https://openreview.net/forum?id=LhO3RdnVOGFi},
}
Figure