Traditional image codecs like JPEG allow users to slide a quality bar from 1 to 100, precisely balancing file size against visual clarity. In contrast, early deep learning-based codecs were rigid: to support five different bitrates, a developer typically had to train, store, and deploy five separate neural networks. This "multi-model" approach is computationally expensive and lacks the fine-grained control required for real-world network fluctuations.
Recent research has converged on a more elegant solution: the variable-rate neural codec. By treating the bitrate as a dynamic input rather than a static training hyperparameter, researchers have developed single-model architectures that can traverse the entire rate-distortion curve on the fly.
The Mechanism of Conditional Scaling
The most intuitive way to control bitrate within a single model is to introduce "knobs" that scale the internal representations of the image. In Variable Rate Deep Image Compression With a Conditional Autoencoder, Yoojin Choi and colleagues introduced a conditional autoencoder that uses two primary variables: the Lagrange multiplier ($\lambda$), which dictates the importance of rate versus distortion during training, and the quantization bin size ($\Delta$). By feeding these values into the network as conditional inputs, the model learns to adjust its behavior based on the desired quality level.
Building on this, Ze Cui and the authors of G-VAE: A Continuously Variable Rate Deep Image Compression Framework proposed the use of "gain units." These units act as simple multipliers that rescale the latent representations before quantization. When the gain is low, more values are pushed toward zero, resulting in a sparser representation that is easier to compress (lower bitrate). By interpolating these gain values exponentially, the framework achieves a smooth, continuous range of quality levels rather than a few discrete steps.
Bridging the Quantization Gap
A significant hurdle in variable-rate compression is the "quantization mismatch." During training, neural networks often use "soft" approximations of quantization (like adding noise) to allow gradients to flow. However, at inference time, they must use "hard" rounding to actual integers.
Zongyu Guo and researchers from USTC addressed this in Soft then Hard: Rethinking the Quantization in Neural Image Compression. They introduced a "Soft-then-Hard" (STH) training strategy that gradually transitions the model from continuous noise to discrete quantization. Combined with Scaled Uniform Noise (SUN), this allows the model to adapt its quantization granularity spatially across an image, ensuring that complex textures get more bits while flat areas get fewer, all within a unified variable-rate framework.
Steering with Prompts and Diffusion
As the field moves toward more complex architectures like Transformers and Diffusion models, the methods for rate control have become increasingly sophisticated. Instead of just scaling latents, researchers are now "steering" models using external cues.
- Visual Prompting: Shiyu Qin and Jinpeng Wang demonstrated in Progressive Learning with Visual Prompt Tuning for Variable-Rate Image Compression that a pre-trained Transformer can be adapted to different bitrates using a "Layer-adaptive Prompt Module" (LPM). By simply changing a small set of learned "prompts" (input tokens), the same frozen backbone can produce images at vastly different quality levels.
- Pseudo Diffusion Timesteps: In the realm of generative compression, Jinpei Guo and the authors of OSCAR found a clever way to repurpose the diffusion process. They mapped different bitrates to "pseudo diffusion timesteps." In this setup, a high-bitrate target corresponds to a "near-final" denoising step, while a low-bitrate target tells the model to perform more aggressive synthesis. This allows a single diffusion-based decoder to handle multiple quality levels in a single step.
Why It Works: Content-Adaptive Granularity
The ultimate goal of these variable-rate systems is to achieve what Anqi Li calls "dynamic granularity adaptation." As detailed in Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaption, a truly flexible codec doesn't just apply a global quality setting; it adjusts the density of information based on the local content of the image. By unifying this spatial adaptivity with a global rate "slider," these models provide a "once-for-all" solution that matches the flexibility of classical codecs while leveraging the superior reconstruction power of deep learning.
As summarized in the Survey on Visual Signal Coding and Processing With Generative Models by Zhibo Chen, the shift from discrete, multi-model codecs to unified, continuously variable architectures is essential for the next generation of visual standards, enabling systems to dynamically respond to bandwidth constraints without the overhead of storing multiple sets of weights.