Image Processing of Pseudo-rate-distortion Function Based on MSSSIM and KL-Divergence, Using Multiple Video Processing Filters for Video Compression

Jinwuk Seok, Seung-Hyun Cho, Hui Yong Kim, Jin Soo ChoiJournal of Broadcast Engineering 2018 · DOI

Abstract

This study introduces a sophisticated video pre-processing framework designed to enhance video compression efficiency. By implementing a "pseudo-rate-distortion" function that utilizes Multi-Scale Structural Similarity (MSSSIM) for quality assessment and Kullback-Leibler (KL) Divergence for bitrate estimation, the proposed system dynamically selects optimal filters for individual 32x32 pixel blocks. Experimental results using the HEVC standard test sequences demonstrate significant bitrate savings of 24% to 64% while maintaining high subjective visual quality, with MSSSIM degradation limited to less than 0.35%.

Introduction

In modern video coding standards like HEVC, the optimization of the rate-distortion (RD) trade-off is paramount. While standard encoders perform RD optimization, they are often constrained by the quality and characteristics of the input source. Pre-processing filters can mitigate these constraints by "cleaning" the video signal before encoding. However, applying a uniform filter across an entire frame often leads to sub-optimal results, as different regions—such as flat backgrounds versus textured edges—require different processing intensities. This paper proposes a block-based selection method using a computationally efficient evaluation function.

Methods

The proposed system employs a block-based pre-processing pipeline. Each 32x32 pixel block is evaluated against a "filter bank" consisting of four options: a heavy-duty downsampling/wavelet filter (sb_vdn1_up), a strong wavelet noise-canceler (as_vdn2), a weak wavelet noise-canceler (as_vdn1), and the original unfiltered block.

The selection is governed by a Pseudo-Rate-Distortion (RD) Function ($J$): $$J = MSSSIM + \lambda \cdot D_{KL}$$

Where $D_{KL}$ (KL-Divergence) serves as a proxy for bitrate by measuring the statistical simplification of DCT coefficients between the original and filtered blocks. The parameter $\lambda$ is a dynamic Lagrange multiplier adjusted via a sigmoid function based on local PSNR, allowing the system to prioritize quality in complex areas and bitrate reduction in simpler ones.

The filter selection architecture, where a selector generates the optimal filter index $n^*$ for each block.

Key Insights and Architecture

A primary innovation is the use of KL-Divergence on the power spectrum of image blocks to estimate bitrate reduction without the need for full, computationally expensive encoding. Furthermore, the authors structure the pre-processing as a multi-layered pipeline that mimics the behavior of Convolutional Neural Networks (CNNs). By using a manually defined filter bank and a mathematically rigorous selection law, the system achieves "intelligent" processing with greater transparency and lower training requirements than traditional deep learning models.

The multi-layered architecture of the pre-processing pipeline, designed to mimic the hierarchical feature processing of neural networks.

Results and Performance

Testing against the HEVC reference software (HM 15.1) on Class B sequences (e.g., Kimono, Park run) revealed substantial performance gains. The method achieved bitrate reductions between 24% and 64%. Crucially, visual quality remained nearly identical to the original, with MSSSIM scores dropping by a negligible 0.09% to 0.35%. Qualitative analysis confirmed that the algorithm intelligently preserves sharp edges by selecting weak or no filters, while aggressively smoothing flat, noisy regions to maximize compression efficiency.

Conclusion

This research demonstrates that a block-based, statistically-driven pre-processing stage can significantly enhance video compression. While the current implementation is an external pre-processing step, future work will focus on reducing computational overhead and integrating the selection logic directly into the encoder's internal processes, such as motion estimation and sub-pixel generation.

Summary updated

BibTeX
@article{seok2018image,
  author = {Jinwuk Seok and Seung-Hyun Cho and Hui Yong Kim and Jin Soo Choi},
  title = {{Image Processing of Pseudo-rate-distortion Function Based on MSSSIM and KL-Divergence, Using Multiple Video Processing Filters for Video Compression}},
  journal = {Journal of Broadcast Engineering},
  year = {2018},
  doi = {10.5909/jbe.2018.23.6.768},
}
Figure