Learned Image Compression with ROI-Weighted Distortion and Bit Allocation

Wei Jiang, Yongqi Zhai, Hangyu Li, Ronggang WangarXiv 2024 · arXiv · OpenReview

TL;DR

This paper introduces TLIC, a learned image compression method that prioritizes perceptual quality by using adversarial training for realistic texture generation and a novel ROI-weighted distortion and bit allocation strategy. This approach aims to allocate more bits to important regions of an image, leading to better visual fidelity where it matters most.

What Problem Does This Paper Address?

Traditional image compression methods often focus on optimizing metrics like Mean Squared Error (MSE), which can lead to visually unappealing artifacts, especially at low bitrates. While recent learned image compression techniques have shown promise, many still prioritize objective metrics over subjective perceptual quality. This paper addresses the challenge of achieving high perceptual quality in learned image compression, particularly for images where certain regions (like faces or text) are more critical than others. The goal is to develop a compression system that can intelligently allocate resources to preserve the visual integrity of these important areas while still achieving efficient compression.

Methods

TLIC builds upon a generative image compression framework. Its core innovations lie in two key areas:

  • Adversarial Training for Realistic Textures: To generate more visually appealing and realistic textures, TLIC employs adversarial loss, similar to Generative Adversarial Networks (GANs). This involves training a discriminator network to distinguish between real and generated images, pushing the generator (the compression model) to produce outputs that are perceptually indistinguishable from the original.
  • ROI-Weighted Distortion and Bit Allocation: This is the paper's main contribution. TLIC uses a Region of Interest (ROI) mask to guide the compression process.
  • ROI Mask Generation: A pre-trained RMformer network is used to generate a saliency map, which serves as the ROI mask. This map highlights areas of interest within the image.
  • Weighted Distortion: The ROI mask is used to control the weight of distortion for each pixel. This means that distortions in ROI regions are penalized more heavily than distortions in non-ROI regions.
  • Weighted Bit Allocation: The ROI mask also influences how bits are allocated. More bits are allocated to regions identified as important (in the ROI) to ensure their high-fidelity reconstruction. This is achieved by adjusting the quantization step size based on the ROI.
  • Variable Rate Control: The method incorporates gain and inverse gain units, inspired by previous work [13], to enable continuous rate adaptation and satisfy target bitrates.
  • Entropy Model: A simplified entropy model is used for efficient entropy coding of the latent representations.

Datasets

Not explicitly detailed in the provided text. The paper mentions using "320 x 320 patches" for training, implying standard image datasets are used, but specific names are not provided.

Key Insights and Ideas

  • Perceptual Quality is Paramount: The paper strongly advocates for prioritizing perceptual quality over purely objective metrics in image compression, especially for content-sensitive images.
  • ROI-Guided Compression is Effective: By explicitly identifying and prioritizing regions of interest, the compression system can achieve significantly better visual results in those critical areas without necessarily sacrificing overall compression efficiency.
  • Adversarial Training Enhances Realism: The use of GAN-like adversarial training is crucial for generating textures that are not just accurate but also visually pleasing and realistic.
  • Synergy of Distortion and Bit Allocation: The dual approach of weighting distortion and allocating bits based on ROI creates a powerful mechanism for targeted quality improvement.

Key Results

The paper states that TLIC aims to achieve better perceptual quality and realistic textures at low bitrates. While specific quantitative results (e.g., PSNR, MS-SSIM, or perceptual metrics) are not presented in the provided text, the authors claim their method is trained with a variable rate compression method and achieves target bitrates. The qualitative benefit is expected to be improved visual fidelity in important image regions.

Limitations and Future Work

  • Pre-trained RMformer: The reliance on a pre-trained RMformer for ROI mask generation means the quality of the ROI detection directly impacts the compression performance.
  • Computational Cost: Adversarial training and complex ROI-based weighting can increase the computational cost of both training and inference.
  • Specific Dataset Details: The lack of explicit dataset information makes it difficult to assess the generalizability of the results.

Future work could involve exploring more adaptive and end-to-end trainable ROI detection mechanisms, further optimizing the adversarial training process for efficiency, and conducting extensive evaluations across a wider range of datasets and perceptual metrics.

Summary updated

Abstract

This short paper describes our method for the track of image compression. To achieve better perceptual quality, we use the adversarial loss to generate realistic textures, use region of interest (ROI) mask to guide the bit allocation for different regions. Our Team name is TLIC.
BibTeX
@misc{jiang2024learned,
  author = {Wei Jiang and Yongqi Zhai and Hangyu Li and Ronggang Wang},
  title = {{Learned Image Compression with ROI-Weighted Distortion and Bit Allocation}},
  year = {2024},
  eprint = {2401.08154},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.08154},
}
Figure