Learning Switchable Priors for Neural Image Compression

Haotian Zhang, Yuqi Li, Dong Liu

Abstract

Neural Image Compression (NIC) traditionally faces a significant trade-off between the complexity of entropy models and computational efficiency. This paper introduces Switchable Priors, a novel framework that decouples entropy model complexity from prior distributions by shifting from parameter estimation to a selection-based paradigm. By predicting an index from a finite set of pre-defined, trainable priors, the proposed method achieves state-of-the-art compression performance while drastically reducing memory overhead and decoding latency. Experimental results on standard benchmarks demonstrate that our lightweight implementation, FastNIC, provides superior BD-rate savings compared to traditional codecs like BPG while maintaining high-speed execution suitable for ultra-high-definition content.

1. Introduction

In Neural Image Compression (NIC), entropy models are critical for estimating the probability distribution of latent variables. Accurate modeling of these distributions—known as priors—is essential for achieving high compression efficiency. However, moving toward more complex distributions, such as Gaussian Mixture Models (GMM) or Generalized Gaussian Models (GGM), typically increases the number of parameters the neural network must predict. Furthermore, standard entropy coding requires the dynamic construction of Cumulative Distribution Function (CDF) tables for every pixel during decoding, a process that is computationally prohibitive for real-time applications. This paper addresses these challenges by proposing a "Switchable Priors" approach that simplifies the entropy modeling process into a discrete selection task.

2. Methodology

2.1 Switchable Priors and Index Prediction

Instead of requiring the entropy model to estimate continuous parameters (e.g., mean $\mu$, scale $\sigma$, or shape $\beta$), the Switchable Priors method utilizes a finite set of $M$ trainable prior distributions. The entropy model is tasked with outputting a continuous value that is subsequently rounded to a discrete index. This index points to a pre-computed CDF table within the prior set, allowing for rapid Look-Up Table (LUT) access during decoding. For more complex models like GMMs, the method extends to multi-dimensional index prediction, where separate indices may represent different distribution components.

2.2 Integrated Skip Mode

A unique feature of this framework is the integration of a "skip mode" directly into the prior set. If a latent variable's contribution to image reconstruction is negligible compared to its bit cost, the model can learn to select a skip index. This allows the system to bypass coding for specific latents, optimizing both the bit rate and computational throughput during inference.

2.3 Training via Soft-to-Hard Assignment

To ensure the model remains differentiable during training, the authors employ a soft-to-hard assignment strategy. During the initial training phases, the model uses a weighted sum of rates (soft assignment) with an annealing temperature. As training progresses, the temperature is adjusted to "harden" the selection into a single discrete index, ensuring that the final inference model operates with discrete LUTs.

2.4 FastNIC Architecture

To demonstrate the practical utility of Switchable Priors, the authors developed FastNIC, a lightweight compression model. FastNIC incorporates efficient architectural components, including FasterNet blocks and partial convolutions, to minimize total complexity while maintaining high reconstruction quality.

3. Experimental Setup

The proposed models were trained on the Flicker2W dataset, consisting of 20,000 high-quality images. Evaluation was conducted across four diverse benchmarks:

  • Kodak: 24 images (768x512), a standard for compression research.
  • Tecnick: 100 images (1200x1200).
  • CLIC 2020: 41 professional-grade images.
  • USTC-TD: 40 ultra-high-definition (4K) images.

4. Results and Discussion

4.1 Compression Efficiency and Speed

FastNIC, powered by Switchable Priors, achieved a 4.10% BD-rate saving over the BPG (HEVC-based) anchor on the Kodak dataset. The model maintains remarkably low complexity, with encoding and decoding requiring only 12 KMACs/pixel and 10 KMACs/pixel, respectively. On a standard GPU, the model can process a 4K image in approximately 100ms, making it viable for high-resolution video and image applications.

4.2 Memory and Resource Optimization

One of the most significant advantages of the Switchable Priors approach is the reduction in memory footprint. Traditional LUT-based GGM models often require over 12,000 CDF tables (~6MB). In contrast, the Switchable Priors method achieves comparable performance using only 40 tables, requiring just 0.01MB of memory. Furthermore, the authors demonstrated that the prior set learned for main latents can be reused for hyperlatents, further optimizing resource usage.

4.3 Versatility

The Switchable Priors module is model-agnostic. When integrated into other state-of-the-art architectures like ELIC or TCM, it consistently reduced entropy coding time and memory requirements without compromising compression gains.

5. Conclusion and Future Work

Switchable Priors represent a paradigm shift in NIC by decoupling distribution complexity from neural network overhead. While the current implementation uses a finite set of 40-50 priors, future research may explore even more complex distribution families that were previously avoided due to the "complexity tax" of parameter estimation. Although the training phase requires careful management of soft assignments, the resulting inference speed and memory efficiency provide a clear path toward deploying advanced neural compression on resource-constrained devices.

Summary updated

Abstract

Neural image compression (NIC) usually adopts a predefined family of probabilistic distributions as the prior of the latent variables, and meanwhile relies on entropy models to estimate the parameters for the probabilistic family. More complex probabilistic distributions may fit the latent variables more accurately, but also incur higher complexity of the entropy models, limiting their practical value. To address this dilemma, we propose a solution to decouple the entropy model complexity from the prior distributions. We use a finite set of trainable priors that correspond to samples of the parametric probabilistic distributions. We train the entropy model to predict the index of the appropriate prior within the set, rather than the specific parameters. Switching between the trained priors further enables us to embrace a skip mode into the prior set, which simply omits a latent variable during the entropy coding. To demonstrate the practical value of our solution, we present a lightweight NIC model, namely FastNIC, together with the learning of switchable priors. FastNIC obtains a better trade-off between compression efficiency and computational complexity for neural image compression. We also implanted the switchable priors into state-of-the-art NIC models and observed improved compression efficiency with a significant reduction of entropy coding complexity.
BibTeX
@article{zhang2025learning,
  author = {Haotian Zhang and Yuqi Li and Li Li and Dong Liu},
  title = {{Learning Switchable Priors for Neural Image Compression}},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  doi = {10.1109/tcsvt.2025.3577134},
}
Figure