Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation

Anqi Li, Feng Li, Yuxi Liu, Runmin Cong, Yao Zhao, Huihui Bai

TL;DR

Control-GIC is a novel generative image compression framework that achieves fine-grained, controllable bitrate adaptation using a single set of model weights. By dynamically adjusting the granularity of image patches based on their local information density, it outperforms state-of-the-art methods in perceptual quality while offering significantly higher efficiency and flexibility.

The Challenge of Flexible Rate Adaptation

Generative Image Compression (GIC) has shown great promise in producing visually stunning reconstructions by prioritizing perceptual realism over pixel-perfect accuracy. However, most GIC models are "fixed-rate." To support different bitrates, developers must train and deploy multiple separate models, which is computationally expensive and highly inflexible for real-world applications.

Existing variable-rate methods often rely on traditional CNN architectures that struggle with perceptual quality at low bitrates or offer only a limited range of rate adjustments. The core challenge lies in creating a single generative model that can adapt its bitrate across a broad spectrum without sacrificing the high-fidelity textures that make generative models so desirable.

Introducing Control-GIC

To overcome these limitations, the authors introduce Control-GIC, built upon a Vector Quantized Generative Adversarial Network (VQGAN) foundation. The framework introduces three main innovations:

1. Granularity-Informed Encoder

Instead of treating the whole image uniformly, the encoder calculates the "entropy" (information density) of local patches. It then sorts these patches and assigns them to one of three hierarchical granularities: fine, medium, or coarse. High-entropy areas (like complex textures) receive more bits, while low-entropy areas (like flat skies) receive fewer.

Entropy-based patch allocation

2. Statistical Entropy Coding

To compress the resulting discrete VQ-indices losslessly, the authors use a non-parametric approach. They capture the frequency distribution of code usage across a large dataset during training to build a global Huffman frequency table. This avoids the overhead of complex, parametric entropy models used in other neural codecs, significantly speeding up the process.

3. Probabilistic Conditional Decoder

This component reconstructs the image by progressively refining features. It uses the transmitted multi-granularity codes as conditions to guide the generation of hierarchical features, ensuring that the final output maintains sharp textures and structural integrity even at lower bitrates.

Key Insights: Dynamic Allocation

The breakthrough insight of Control-GIC is treating bitrate control as a resource allocation problem. By adjusting the ratio of fine-to-coarse patches ($r_1, r_2, r_3$), users can tune the bitrate with extreme precision (down to 0.001 bpp increments) during inference without retraining the model.

Because VQGAN represents images as sequences of discrete codes, there is a direct, linear correlation between the number of codes (granularity) and the final bitrate. Unlike standard Rate-Distortion (R-D) optimization that bakes a specific bitrate into the model via a Lagrange multiplier ($\lambda$), Control-GIC's weights are optimized for general reconstruction. The rate control is entirely decoupled and handled by the structural decisions made by the encoder.

Control-GIC vs. Fixed and Variable-rate Methods

Performance and Results

Evaluated on standard benchmarks (Kodak, DIV2K, CLIC2020), Control-GIC demonstrates significant advantages:

  • Perceptual Superiority: It consistently outperforms traditional codecs (BPG, VVC) and recent generative models (HiFiC, MRIC) in perceptual metrics like LPIPS and FID across various bitrates.
  • Speed and Efficiency: The non-parametric entropy coding makes the model remarkably fast, achieving 7x faster encoding and 3x faster decoding than MS-ILLM, making it highly practical for real-world deployment.
  • Fine-Grained Control: The model smoothly transitions between bitrates while maintaining better texture integrity and sharpness than competitors, which often produce blurs or misleading artifacts at lower rates.

Limitations and Future Work

While Control-GIC handles general textures exceptionally well, the authors note that reconstructing very small faces remains a challenge. Even with fine granularity, minute facial features can sometimes lose detail. Future work could involve semantic-aware granularity allocation to better handle these high-importance regions. Additionally, exploring hybrid approaches that combine the speed of non-parametric entropy coding with the density of learned models could further push the boundaries of compression ratios.

Summary updated

Abstract

Although recent generative image compression methods have demonstrated impressive potential in optimizing the rate-distortion-perception trade-off, they still face the critical challenge of flexible rate adaptation to diverse compression necessities and scenarios. To overcome this challenge, this paper proposes a $\textbf{Control}$lable $\textbf{G}$enerative $\textbf{I}$mage $\textbf{C}$ompression framework, $\textbf{Control-GIC}$, the first capable of fine-grained bitrate adaptation across a broad spectrum while ensuring high-fidelity and generality compression. We base Control-GIC on a VQGAN framework representing an image as a sequence of variable-length codes ($\textit{i.e.}$ VQ-indices), which can be losslessly compressed and exhibits a direct positive correlation with bitrates. Drawing inspiration from the classical coding principle, we correlate the information density of local image patches with their granular representations. Hence, we can flexibly determine a proper allocation of granularity for the patches to achieve dynamic adjustment for VQ-indices, resulting in desirable compression rates. We further develop a probabilistic conditional decoder capable of retrieving historic encoded multi-granularity representations according to transmitted codes, and then reconstruct hierarchical granular features in the formalization of conditional probability, enabling more informative aggregation to improve reconstruction realism. Our experiments show that Control-GIC allows highly flexible and controllable bitrate adaptation where the results demonstrate its superior performance over recent state-of-the-art methods.
BibTeX
@inproceedings{li2025once,
  author = {Anqi Li and Feng Li and Yuxi Liu and Runmin Cong and Yao Zhao and Huihui Bai},
  title = {{Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=z0hUsPhwUN},
}
Figure