TL;DR
This paper tackles the memory inefficiency of learned image compression (LIC) models, which hinders their deployment on resource-constrained devices. The authors propose a novel pruning method, ERHP, specifically designed to optimize the hyperprior module, a key component in state-of-the-art LIC. Their approach effectively reduces model size while simultaneously improving compression performance.
What Problem Does This Paper Address?
Learned Image Compression (LIC) has emerged as a powerful alternative to traditional image compression techniques, achieving superior rate-distortion performance. A significant contributor to this success is the "hyperprior" module, which captures spatial redundancies in the latent representation of an image. However, these hyperprior-based LIC models are notoriously memory-intensive, making them impractical for deployment on devices like smartphones or edge computing platforms. The authors identify that the hyperprior module is often over-parameterized and contains redundant information, leading to excessive memory footprints. This paper aims to address this critical bottleneck by developing a method to efficiently prune the hyperprior module without sacrificing, and even improving, compression quality.
Methods
The core of this paper is the proposed ERHP (Enhanced Resrep on Hyper Path) pruning method. ERHP builds upon the ResRep pruning technique but adapts it for the specific needs of LIC models, particularly focusing on the hyperprior module.
Here's a breakdown of their approach:
- Identifying Over-parameterization: The authors first analyze the hyperprior module and demonstrate that it is significantly over-parameterized. They show that even reducing the number of channels in the hyperprior module by half has minimal impact on performance, confirming the presence of redundancy.
- Adapting ResRep: The ResRep method typically prunes standard convolutional layers by introducing a "compactor" layer. ERHP extends this by incorporating compactors for layers commonly found in LIC hyperprior modules but not directly supported by the original ResRep:
- PixelShuffle Layers: These layers are used to upsample feature maps. ERHP places the compactor after the shuffling operation to effectively prune channels that contribute to the upsampling.
- Deconvolution Layers: These layers are used in the decoder part of the hyperprior path. ERHP adapts the compactor to work with the transposed channel structure of deconvolution layers.
- Pruning Strategy: ERHP applies pruning specifically to the hyperpath (encoder and decoder of the hyperprior module) while keeping the main path (encoder and decoder of the image latent representation) frozen. This strategy aims to reduce the memory cost associated with the hyperprior module, which is crucial for capturing the necessary side information.
- Finetuning: After pruning, the authors finetune the pruned network. This step is critical because image compression tasks require higher quality than tasks like image classification (where ResRep was originally proposed). Finetuning helps recover any performance degradation caused by pruning.
Datasets
- Training: The paper mentions training their models on OpenImage [19].
- Evaluation: Models are tested on the Kodak [20] dataset.
Key Insights and Ideas
- Hyperprior Module is a Bottleneck: The paper convincingly demonstrates that the hyperprior module, while crucial for performance, is a major contributor to the memory overhead of LIC models due to its over-parameterization.
- Targeted Pruning is Effective: By focusing pruning efforts specifically on the hyperprior module and adapting pruning techniques to its unique layer types (PixelShuffle, deconvolution), ERHP achieves significant memory reduction.
- Performance Improvement is Possible: Counter-intuitively, pruning the hyperprior module, when done correctly and followed by finetuning, can lead to improved rate-distortion performance. This suggests that removing redundant information in the hyperprior can actually lead to a more efficient and effective compression process.
- Finetuning is Essential for Compression: The authors highlight that finetuning is a necessary step for image compression tasks, as they have higher quality requirements compared to classification tasks.
Key Results
The paper reports significant improvements in memory efficiency and compression performance:
- Parameter Reduction: ERHP achieves at least 22.6% parameter reduction in the whole model. For the Hyperprior model, they report up to 33.1% reduction in the hyperprior module.
- Performance Improvement: In several cases, ERHP not only matches but also slightly improves the rate-distortion performance compared to the original models. For instance, Table 1 shows instances where the pruned Hyperprior model achieves better PSNR at a lower bitrate.
- Comparison to Manual Pruning: ERHP consistently outperforms manually pruned models at similar parameter scales, demonstrating the effectiveness of their automated and targeted approach.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the context model (if present) and the main path are frozen during pruning. This means that the potential for further optimization by pruning these parts is not explored.
- While ERHP adapts ResRep for PixelShuffle and deconvolution layers, it's possible that other specialized layers in advanced LIC architectures might require further adaptation.
- Future Work:
- The authors suggest exploring pruning the main path and context model to achieve even greater memory savings.
- Investigating the application of ERHP to other advanced LIC architectures beyond Hyperprior and Cheng is a natural next step.
- Further research into more sophisticated pruning strategies that can dynamically adapt to different layer types and model architectures could be beneficial.
Summary updated