Sensitivity Decouple Learning for Image Compression Artifacts Reduction

Li Ma, Yifan Zhao, Peixi Peng, Yonghong Tian

Abstract

Lossy image compression, while essential for efficient data storage and transmission, introduces undesirable artifacts that degrade visual quality and hinder downstream computer vision tasks. Traditional restoration methods often treat artifact reduction as a direct pixel-to-pixel mapping, potentially neglecting the intrinsic semantic attributes of the image. This paper introduces Sensitivity Decouple Learning (SDL), a novel framework that decouples image features into compression-insensitive (high-level semantics) and compression-sensitive (low-level quality cues) components. By integrating these features through a Dual Awareness Guidance Network (DAGN), the proposed method achieves state-of-the-art restoration performance and significantly improves the accuracy of machine vision tasks on compressed data.

Introduction

Digital media relies heavily on lossy compression standards like JPEG to manage bandwidth. However, at low bitrates, these methods produce noticeable artifacts—blocky patterns and blurred edges. While deep learning models have improved restoration, many act as "black boxes" that may inadvertently smooth out critical semantic information. This creates a "vision gap" where restored images might look acceptable to humans but remain difficult for AI systems to interpret. The authors of "Sensitivity Decouple Learning for Image Compression Artifacts Reduction" propose addressing this by explicitly modeling how different image attributes respond to compression.

The Challenge of Compression Artifacts

When an image is compressed, global structures often remain intact while local textures are lost. For instance, in a compressed image of a golf ball, the overall spherical shape (a high-level semantic feature) is preserved, but the fine dimple textures (low-level details) are obscured by blocking artifacts. This discrepancy highlights the need for a restoration approach that respects the underlying content while repairing local distortions.

Visualizing the impact of low-quality compression on local textures versus global shape.

Methodology: Sensitivity Decouple Learning

The core innovation of this work is the decoupling of features based on their sensitivity to compression. The proposed Dual Awareness Guidance Network (DAGN) utilizes two specialized paths to process internal information:

  1. Compression-Insensitive Features: These represent the "essence" or high-level semantics of the image (e.g., identifying a face or a hat). Adversarial training is employed to ensure these features remain consistent between compressed and original images, preserving the information necessary for machine vision.
  2. Compression-Sensitive Features: These focus on the degradation itself, capturing edges and shapes affected by the specific Quality Factor (QF). The model is trained to predict the compression degree, making the network "aware" of the required restoration intensity.
Architecture of the Dual Awareness Guidance Network (DAGN).

These two streams are merged via a Cross-Feature Fusion Module (CFM) into the main Artifacts Reduction Pipeline, ensuring the final output is both visually sharp and semantically accurate.

Experimental Results

The DAGN was evaluated using industry-standard benchmarks, including DIV2K and Flickr2K for training, and LIVE1, BSD500, and LIU4K for evaluation.

  • Quantitative Performance: The model achieved a significant average 2.06 dB PSNR gain on the BSD500 dataset compared to baseline compressed images.
  • Machine Vision Impact: On the MS COCO dataset at QF=10, DAGN restoration boosted object detection accuracy (mAP) from 9.5% (raw JPEG) to 26.1%, demonstrating its effectiveness for downstream AI tasks.
  • Efficiency: Designed for practical use, the model processes a 512x512 image in approximately 29.7 ms, making it viable for real-time applications.

Conclusion

Sensitivity Decouple Learning offers a robust approach to image restoration by acknowledging the dual nature of image features. By protecting semantic integrity while reconstructing lost details, DAGN bridges the gap between human perception and machine vision. The authors suggest that this decoupling philosophy could be extended to other inverse problems, such as image denoising and super-resolution, where balancing global semantics and local restoration is equally critical.

Summary updated

Abstract

With the benefit of deep learning techniques, recent researches have made significant progress in image compression artifacts reduction. Despite their improved performances, prevailing methods only focus on learning a mapping from the compressed image to the original one but ignore the intrinsic attributes of the given compressed images, which greatly harms the performance of downstream parsing tasks. Different from these methods, we propose to decouple the intrinsic attributes into two complementary features for artifacts reduction, i.e., the compression-insensitive features to regularize the high-level semantic representations during training and the compression-sensitive features to be aware of the compression degree. To achieve this, we first employ adversarial training to regularize the compressed and original encoded features for retaining high-level semantics, and we then develop the compression quality-aware feature encoder for compression-sensitive features. Based on these dual complementary features, we propose a Dual Awareness Guidance Network (DAGN) to utilize these awareness features as transformation guidance during the decoding phase. In our proposed DAGN, we develop a cross-feature fusion module to maintain the consistency of compression-insensitive features by fusing compression-insensitive features into the artifacts reduction baseline. Our method achieves an average 2.06 dB PSNR gains on BSD500, outperforming state-of-the-art methods, and only requires 29.7 ms to process one image on BSD500. Besides, the experimental results on LIVE1 and LIU4K also demonstrate the efficiency, effectiveness, and superiority of the proposed method in terms of quantitative metrics, visual quality, and downstream machine vision tasks.
BibTeX
@article{ma2024sensitivity,
  author = {Li Ma and Yifan Zhao and Peixi Peng and Yonghong Tian},
  title = {{Sensitivity Decouple Learning for Image Compression Artifacts Reduction}},
  journal = {IEEE Trans. Image Process.},
  year = {2024},
  doi = {10.1109/tip.2024.3403034},
}
Figure