Learned Image Compression With Super-Resolution Residual Modules and DISTS Optimization

Akifumi Suzuki, Hiroaki Akutsu, Takahiro Naruko, Koki Tsubota, Kiyoharu AizawaWorkshops @ CVPR 2021 · OpenReview

Abstract

Neural network-based image compressors have the ability to optimize various perceptual image quality metrics. We propose improved methods based on selective-detail decoding, which uses two decoders (a main decoder and selective-detail decoder) optimized for different image-quality metrics and applies the output result of a suitable decoder for each part of an image. The following three improvements are obtained: (1) we add a super-resolution residual module to the main decoder to output visually clearer images; (2) we use an image quality metric based on Deep Image Structure and Texture Similarity (DISTS) to improve perceptual quality; (3) we use cross-entropy loss for mask accuracy and weighted mean squared error for text visual quality.

Introduction

In image compression, a fundamental trade-off exists between bitrate and visual quality. Traditional codecs like JPEG and BPG often suffer from texture blurring or blocky artifacts at ultra-low bitrates. While standard neural compression models often optimize for Mean Squared Error (MSE), this metric frequently fails to align with human visual perception, particularly regarding high-frequency details like sharp text edges or fine skin textures. This paper introduces a selective-detail decoding framework designed to maintain sharpness and texture fidelity even at extremely low bitrates.

Proposed Architecture

The core of the proposed system is a selective-detail decoding architecture. Instead of a single monolithic decoder, the framework employs two specialized components: a Main Decoder ($G_m$) for general image structure and texture, and a Selective-Detail Decoder ($G_s$) focused on high-priority regions such as faces and text.

Overall architecture of the proposed selective-detail decoding framework.

The pipeline begins with an encoder $E$ that transforms the input image $x$ into a latent representation $z$. This latent is quantized ($Q$) and entropy-coded. On the decoding side, the latent is split into $\hat{z}_m$ and $\hat{z}_s$ for the respective decoders. A mask generator determines which decoder's output is used for specific image regions, ensuring that critical details are preserved by the selective-detail decoder.

Key Innovations

Super-Resolution Residual Module (SRRM)

To enhance the visual clarity of the main decoder's output, the authors introduced the Super-Resolution Residual Module (SRRM). Inspired by super-resolution tasks, this module forces the network to learn edge restoration.

Detailed structure of the Super-Resolution Residual Module (SRRM).

As shown in the SRRM architecture, the module takes the output of the main decoder $\hat{x}_m$, upsamples it by a factor of 2, processes it through a decoder body and convolutional layers to generate a high-resolution residual, and then downsamples the result back to the original resolution. This "upsample-downsample" process encourages the model to reconstruct sharp edges that are typically lost during low-bitrate compression.

DISTS Optimization and Improved Masking

Beyond structural changes, the model optimizes for Deep Image Structure and Texture Similarity (DISTS). Unlike MSE, DISTS leverages deep features to capture how humans perceive texture similarity, leading to more natural-looking reconstructions.

Furthermore, to improve the accuracy of the decoder selection mask, the authors utilized cross-entropy loss to compare predicted masks against ground truth semantic labels (e.g., for faces and text). A weighted MSE loss was also applied to prioritize the visual quality of text-heavy regions, ensuring legibility at low bitrates.

Results and Discussion

The model was trained on the Open Images Challenge 2018 dataset and evaluated using the CLIC2021 validation set. Performance was compared against JPEG and BPG at target bitrates of 0.075, 0.15, and 0.30 bits per pixel (bpp).

Quantitatively, the proposed method significantly outperformed traditional codecs in perceptual metrics. At 0.075 bpp, the method achieved a DISTS score of 0.185, compared to 0.231 for BPG and 0.409 for JPEG (lower is better). Qualitative analysis confirmed that the SRRM effectively reduces artifacts in areas with sharp intensity transitions, such as the edges of objects or text characters.

Conclusion

By combining selective-detail decoding with super-resolution techniques and perceptually-aligned loss functions, the proposed framework achieves superior visual quality at ultra-low bitrates. Future work may address the computational overhead of the dual-decoder system and explore more extensive human subjective testing to further validate the DISTS-based improvements.

Summary updated

Abstract

Neural network-based image compressors have the ability to optimize various perceptual image quality metrics. We propose improved methods that is based on selective-detail decoding, which uses two decoders (a main decoder and selective-detail decoder) optimized for different image-quality metrics and applies the output result of a suitable decoder for each part of an image. The following three improvements are obtained with the proposed method. (1) Inspired by the super-resolution task, we add a super-resolution residual module to the main decoder, which is trained to up-sample an image to a resolution beyond the source image, aiming to output a visually clearer image. (2) To improve the perceptual image quality of the main decoder, we use an image quality metric based on Deep Image Structure and Texture Similarity (DISTS), the similarity of which is close to that of human senses with respect to texture. (3) To improve the mask accuracy for decoder selection, cross entropy loss is used for comparing predicted masks and ground truth masks. We also use the weighted mean squared error to improve the visual quality of the text part of an image.
BibTeX
@inproceedings{suzuki2021learned,
  author = {Akifumi Suzuki and Hiroaki Akutsu and Takahiro Naruko and Koki Tsubota and Kiyoharu Aizawa},
  title = {{Learned Image Compression With Super-Resolution Residual Modules and DISTS Optimization}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  url = {https://openreview.net/forum?id=hk9XmGg9ph6},
}
Figure