Abstract
Neural network-based image compressors have the ability to optimize various perceptual image quality metrics. We propose improved methods based on selective-detail decoding, which uses two decoders (a main decoder and selective-detail decoder) optimized for different image-quality metrics and applies the output result of a suitable decoder for each part of an image. The following three improvements are obtained: (1) we add a super-resolution residual module to the main decoder to output visually clearer images; (2) we use an image quality metric based on Deep Image Structure and Texture Similarity (DISTS) to improve perceptual quality; (3) we use cross-entropy loss for mask accuracy and weighted mean squared error for text visual quality.
Introduction
In image compression, a fundamental trade-off exists between bitrate and visual quality. Traditional codecs like JPEG and BPG often suffer from texture blurring or blocky artifacts at ultra-low bitrates. While standard neural compression models often optimize for Mean Squared Error (MSE), this metric frequently fails to align with human visual perception, particularly regarding high-frequency details like sharp text edges or fine skin textures. This paper introduces a selective-detail decoding framework designed to maintain sharpness and texture fidelity even at extremely low bitrates.
Proposed Architecture
The core of the proposed system is a selective-detail decoding architecture. Instead of a single monolithic decoder, the framework employs two specialized components: a Main Decoder ($G_m$) for general image structure and texture, and a Selective-Detail Decoder ($G_s$) focused on high-priority regions such as faces and text.
The pipeline begins with an encoder $E$ that transforms the input image $x$ into a latent representation $z$. This latent is quantized ($Q$) and entropy-coded. On the decoding side, the latent is split into $\hat{z}_m$ and $\hat{z}_s$ for the respective decoders. A mask generator determines which decoder's output is used for specific image regions, ensuring that critical details are preserved by the selective-detail decoder.
Key Innovations
Super-Resolution Residual Module (SRRM)
To enhance the visual clarity of the main decoder's output, the authors introduced the Super-Resolution Residual Module (SRRM). Inspired by super-resolution tasks, this module forces the network to learn edge restoration.
As shown in the SRRM architecture, the module takes the output of the main decoder $\hat{x}_m$, upsamples it by a factor of 2, processes it through a decoder body and convolutional layers to generate a high-resolution residual, and then downsamples the result back to the original resolution. This "upsample-downsample" process encourages the model to reconstruct sharp edges that are typically lost during low-bitrate compression.
DISTS Optimization and Improved Masking
Beyond structural changes, the model optimizes for Deep Image Structure and Texture Similarity (DISTS). Unlike MSE, DISTS leverages deep features to capture how humans perceive texture similarity, leading to more natural-looking reconstructions.
Furthermore, to improve the accuracy of the decoder selection mask, the authors utilized cross-entropy loss to compare predicted masks against ground truth semantic labels (e.g., for faces and text). A weighted MSE loss was also applied to prioritize the visual quality of text-heavy regions, ensuring legibility at low bitrates.
Results and Discussion
The model was trained on the Open Images Challenge 2018 dataset and evaluated using the CLIC2021 validation set. Performance was compared against JPEG and BPG at target bitrates of 0.075, 0.15, and 0.30 bits per pixel (bpp).
Quantitatively, the proposed method significantly outperformed traditional codecs in perceptual metrics. At 0.075 bpp, the method achieved a DISTS score of 0.185, compared to 0.231 for BPG and 0.409 for JPEG (lower is better). Qualitative analysis confirmed that the SRRM effectively reduces artifacts in areas with sharp intensity transitions, such as the edges of objects or text characters.
Conclusion
By combining selective-detail decoding with super-resolution techniques and perceptually-aligned loss functions, the proposed framework achieves superior visual quality at ultra-low bitrates. Future work may address the computational overhead of the dual-decoder system and explore more extensive human subjective testing to further validate the DISTS-based improvements.
Summary updated