On Efficient Neural Network Architectures for Image Compression

Yichi Zhang, Zhihao Duan, Fengqing ZhuICIP 2024 · DOI

TL;DR

This paper tackles the challenge of creating efficient neural network architectures for image compression, which often sacrifice speed for high compression quality. By systematically studying the impact of different transform and context models, the authors identify key architectural components that lead to a superior Rate-Distortion-Complexity (RDC) trade-off, culminating in a model that outperforms existing methods while being significantly less computationally demanding.

What Problem Does This Paper Address?

Image compression is crucial for managing the ever-increasing volume of visual data. While traditional methods like JPEG and HEVC have been refined over decades, recent deep learning-based image compression (LIC) techniques have shown remarkable performance, sometimes even surpassing these established standards. However, a major hurdle for LIC is its computational complexity, making it impractical for many real-world applications. This paper addresses the critical need for LIC models that achieve a favorable balance between compression efficiency (rate-distortion) and computational cost (complexity), a trade-off known as the RDC trade-off. The authors aim to provide insights into designing efficient LIC architectures by dissecting the problem into its core components: transforms and context models.

Methods

The authors conduct a comprehensive empirical study by breaking down the LIC framework into two main parts:

  • Transforms: These modules are responsible for decorrelating and compressing the image data into a latent representation. The study explores various existing network architectures, including convolutional neural networks (CNNs) like ConvNeXt and DCNv4, and transformer-based models like Swin Transformer and EfficientFormerV2, evaluating their effectiveness as image transforms for compression. They also investigate attention mechanisms and deformable convolutions.
  • Context Models: These models enhance coding efficiency by predicting the probability distribution of latent variables, leveraging spatial and/or channel dependencies. The paper examines several context models:
  • Hierarchical: Generalizing autoregressive models.
  • Channel-wise Autoregressive (CHARM): Models channel relationships.
  • Space-Channel Context Model (SCCTX): Combines spatial and channel dependencies, incorporating an uneven channel grouping strategy.
  • Quantization-Aware ResNet VAE (QARV): A hierarchical model.

The study adapts a baseline M&S Hyperprior model and systematically replaces its transform and context modules with various architectures. Complexity is measured by the number of parameters, while performance is evaluated using BD-Rate (for rate-distortion) and encoding/decoding times. A novel metric, latent correlation ($p_{k \times k}$), is introduced to quantify the de-correlation ability of different transforms.

Datasets

  • Training: COCO 2017 Trainset (118,287 images, 640x420 pixels). Patches of 256x256 were randomly cropped for training.
  • Evaluation: Kodak, Tecnick, and CLIC 2022 datasets.

Key Insights and Ideas

  • De-correlation is Key for Transforms: The study strongly suggests that the ability of a transform module to effectively de-correlate the latent space is paramount for achieving good compression performance. Lower latent correlation ($p_{k \times k}$) directly correlates with better BD-Rate.
  • Synergy of Transforms and Context Models: The best RDC trade-off is achieved not just by optimizing individual components but by finding synergistic combinations. For instance, advanced context models like SCCTX significantly boost performance when paired with powerful transforms like DCNv4.
  • DCNv4 and NAFNet Shine: Among the tested architectures, DCNv4 (for its strong feature extraction from large-scale vision tasks) and NAFNet (for its proficiency in image restoration) emerged as top-performing transforms, offering a good balance of complexity and performance.
  • SCCTX Outperforms Other Context Models: The Space-Channel Context Model (SCCTX) consistently outperformed CHARM and QARV, highlighting the benefits of its combined spatial and channel dependency modeling, especially with uneven channel information.
  • Complexity vs. Performance Trade-off: The paper demonstrates that it's possible to achieve state-of-the-art compression performance with significantly fewer parameters and faster inference times compared to existing methods.

Key Results

  • The proposed final model, DCNv4 SCCTX (Model E), achieves an exceptional RDC trade-off, yielding -4.10% BD-Rate on the Kodak dataset while having significantly lower complexity than methods like STF.
  • NAFNet HP (Model B) and DCNv4 HP (Model C), when used as transforms, significantly outperform previous methods like Cheng2020 HP, with DCNv4 HP achieving 0.56% BD-Rate and having only 27.90M parameters compared to Cheng2020's 115.3M.
  • The SCCTX context model, when applied to DCNv4 HP, resulted in DCNv4 SCCTX (Model E), achieving -4.10% BD-Rate on Kodak, -5.91% on Tecnick, and -5.60% on CLIC2022, outperforming STF despite STF's much larger parameter count (599.1M vs. 44.95M).
  • Encoding and decoding times for the proposed models are competitive with existing methods, even for models with more complex context modules.

Limitations and Future Work

  • Lack of Theoretical Analysis: The study primarily relies on empirical investigation and lacks a deep theoretical analysis of the rate-distortion-complexity trade-off.
  • Complexity Reduction Techniques: The authors did not explore other methods for reducing complexity, such as knowledge distillation, model quantization, and pruning. These are identified as promising avenues for future work.
  • Further Exploration: The paper suggests that future research could delve deeper into exploring novel transform and context model designs, as well as investigating these techniques for video compression.

Summary updated

BibTeX
@inproceedings{zhang2024efficient,
  author = {Yichi Zhang and Zhihao Duan and Fengqing Zhu},
  title = {{On Efficient Neural Network Architectures for Image Compression}},
  booktitle = {ICIP},
  year = {2024},
  doi = {10.1109/icip51287.2024.10647669},
}
Figure