TL;DR
This paper tackles the challenge of deploying learned image compression (LIC) on resource-constrained devices by quantizing network weights to 8-bit fixed-point. They introduce novel grouping and quantization strategies, along with a weight clipping fine-tuning method, to minimize coding loss and achieve significant model size reduction while outperforming traditional codecs like BPG.
What Problem Does This Paper Address?
Learned image compression (LIC) has shown remarkable performance, often surpassing traditional methods like JPEG2000 and BPG in terms of coding gain. However, the deep neural networks underpinning LIC typically employ 32-bit floating-point weights, leading to large model sizes. This makes them impractical for deployment on embedded systems with limited memory and computational power. The core problem this paper addresses is bridging this gap: enabling efficient, high-performance image compression on resource-constrained devices by drastically reducing the model size of LIC networks without sacrificing significant compression quality.
Methods
The authors propose a multi-pronged approach to quantize LIC networks to 8-bit fixed-point weights:
- Weight Quantization: They quantize weights to 8-bit fixed-point, representing them as
{1, IL, FL}(signed bit, integer bits, fractional bits). They specifically setIL=1andFL=6. - Grouped Quantization: Instead of quantizing each weight individually, they group weights. This allows for a scaling factor (
sfk) to be applied to each group, reducing the number of parameters to store. - Non-linear Memory-Free Codebook (NLQ): They develop a novel, memory-free codebook for quantization. This codebook is derived from the weight distribution and avoids the need for large lookup tables (LUTs) at runtime, crucial for embedded systems.
- Optimal Grouping and Quantization Scheme: They explore two grouping strategies: layer-wise (LW) and channel-wise (CW). Through experimentation, they find that channel-wise grouping combined with their NLQ scheme (CW-NLQ) yields the best coding gain.
- Weight Clipping Fine Tuning (WCFT): To mitigate the coding loss introduced by quantization, especially at higher rates, they propose a weight clipping fine-tuning method. This involves clipping the maximum weight magnitude and then fine-tuning the network using a straight-through estimator to preserve gradients.
Datasets
- ImageNet: Used for training the network. The authors mention cropping 256x256 patches from ImageNet for training.
- Kodak dataset: Used for evaluating the coding performance of their proposed method.
Key Insights and Ideas
- Grouped Quantization is Key: Quantizing weights in groups, rather than individually, is essential for reducing model size while managing the overhead of scaling factors.
- Channel-wise Grouping Wins: The channel-wise grouping strategy proved more effective than layer-wise grouping for achieving better coding gains.
- Memory-Free Codebooks are Practical: The proposed non-linear, memory-free codebook is a critical innovation for embedded deployment, as it eliminates the need for large LUTs.
- Fine-tuning Recovers Lost Quality: The weight clipping fine-tuning (WCFT) method is crucial for recovering coding performance lost due to quantization, particularly at higher compression rates.
- 8-bit is Sufficient: The research demonstrates that 8-bit fixed-point quantization can achieve competitive results with minimal loss compared to 32-bit floating-point models.
Key Results
- Model Size Reduction: Achieved approximately 75% model size reduction compared to the 32-bit floating-point anchor model.
- Coding Performance: The proposed 8-bit fixed-point LIC method is close to the original 32-bit floating-point version. For middle rate models, BD-PSNR loss is around 0.1183dB (MSE) and 0.1486dB (MS-SSIM).
- Outperforms BPG: The quantized LIC method outperforms BPG in terms of MS-SSIM.
- WCFT Effectiveness: WCFT significantly reduces coding loss. For high-rate MSE models, it reduces BD-PSNR loss from 0.382dB to 0.148dB and bit increment from 0.019 to 0.004. Similar improvements are seen for MS-SSIM.
Limitations and Future Work
- Limitations: The paper focuses on weight quantization. While they achieve significant size reduction, the activation functions are still processed using floating-point arithmetic. The coding loss, though small, is still present, especially at higher rates, and WCFT is used to mitigate this.
- Future Work: The authors suggest quantizing activations using fixed-point arithmetic and designing corresponding hardware architectures for FPGAs and ASICs. This would further enhance the efficiency and suitability for embedded systems.
Summary updated