TL;DR
Control-GIC is a novel generative image compression framework that achieves fine-grained, controllable bitrate adaptation using a single set of model weights. By dynamically adjusting the granularity of image patches based on their local information density, it outperforms state-of-the-art methods in perceptual quality while offering significantly higher efficiency and flexibility.
The Challenge of Flexible Rate Adaptation
Generative Image Compression (GIC) has shown great promise in producing visually stunning reconstructions by prioritizing perceptual realism over pixel-perfect accuracy. However, most GIC models are "fixed-rate." To support different bitrates, developers must train and deploy multiple separate models, which is computationally expensive and highly inflexible for real-world applications.
Existing variable-rate methods often rely on traditional CNN architectures that struggle with perceptual quality at low bitrates or offer only a limited range of rate adjustments. The core challenge lies in creating a single generative model that can adapt its bitrate across a broad spectrum without sacrificing the high-fidelity textures that make generative models so desirable.
Introducing Control-GIC
To overcome these limitations, the authors introduce Control-GIC, built upon a Vector Quantized Generative Adversarial Network (VQGAN) foundation. The framework introduces three main innovations:
1. Granularity-Informed Encoder
Instead of treating the whole image uniformly, the encoder calculates the "entropy" (information density) of local patches. It then sorts these patches and assigns them to one of three hierarchical granularities: fine, medium, or coarse. High-entropy areas (like complex textures) receive more bits, while low-entropy areas (like flat skies) receive fewer.
2. Statistical Entropy Coding
To compress the resulting discrete VQ-indices losslessly, the authors use a non-parametric approach. They capture the frequency distribution of code usage across a large dataset during training to build a global Huffman frequency table. This avoids the overhead of complex, parametric entropy models used in other neural codecs, significantly speeding up the process.
3. Probabilistic Conditional Decoder
This component reconstructs the image by progressively refining features. It uses the transmitted multi-granularity codes as conditions to guide the generation of hierarchical features, ensuring that the final output maintains sharp textures and structural integrity even at lower bitrates.
Key Insights: Dynamic Allocation
The breakthrough insight of Control-GIC is treating bitrate control as a resource allocation problem. By adjusting the ratio of fine-to-coarse patches ($r_1, r_2, r_3$), users can tune the bitrate with extreme precision (down to 0.001 bpp increments) during inference without retraining the model.
Because VQGAN represents images as sequences of discrete codes, there is a direct, linear correlation between the number of codes (granularity) and the final bitrate. Unlike standard Rate-Distortion (R-D) optimization that bakes a specific bitrate into the model via a Lagrange multiplier ($\lambda$), Control-GIC's weights are optimized for general reconstruction. The rate control is entirely decoupled and handled by the structural decisions made by the encoder.
Performance and Results
Evaluated on standard benchmarks (Kodak, DIV2K, CLIC2020), Control-GIC demonstrates significant advantages:
- Perceptual Superiority: It consistently outperforms traditional codecs (BPG, VVC) and recent generative models (HiFiC, MRIC) in perceptual metrics like LPIPS and FID across various bitrates.
- Speed and Efficiency: The non-parametric entropy coding makes the model remarkably fast, achieving 7x faster encoding and 3x faster decoding than MS-ILLM, making it highly practical for real-world deployment.
- Fine-Grained Control: The model smoothly transitions between bitrates while maintaining better texture integrity and sharpness than competitors, which often produce blurs or misleading artifacts at lower rates.
Limitations and Future Work
While Control-GIC handles general textures exceptionally well, the authors note that reconstructing very small faces remains a challenge. Even with fine granularity, minute facial features can sometimes lose detail. Future work could involve semantic-aware granularity allocation to better handle these high-importance regions. Additionally, exploring hybrid approaches that combine the speed of non-parametric entropy coding with the density of learned models could further push the boundaries of compression ratios.
Summary updated