Practical Full Resolution Learned Lossless Image Compression

Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, Luc Van Gool

Abstract

We propose the first practical learned lossless image compression system, L3C, and show that it outperforms the popular engineered codecs, PNG, WebP and JPEG 2000. At the core of our method is a fully parallelizable hierarchical probabilistic model for adaptive entropy coding which is optimized end-to-end for the compression task. In contrast to recent autoregressive discrete probabilistic models such as PixelCNN, our method i) models the image distribution jointly with learned auxiliary representations instead of exclusively modeling the image distribution in RGB space, and ii) only requires three forward-passes to predict all pixel probabilities instead of one for each pixel. As a result, L3C obtains over two orders of magnitude speedups when sampling compared to the fastest PixelCNN variant (Multiscale-PixelCNN). Furthermore, we find that learning the auxiliary representation is crucial and outperforms predefined auxiliary representations such as an RGB pyramid significantly.

Introduction

Lossless image compression requires an exact reconstruction of every pixel. Traditionally, this is achieved using hand-engineered codecs like PNG, WebP, or JPEG 2000. Recently, generative models from the deep learning community, such as PixelCNN, have demonstrated the ability to predict pixel distributions with high accuracy, theoretically enabling superior compression ratios.

However, these models are typically autoregressive, predicting one pixel at a time based on all previously processed pixels. For high-resolution images, this necessitates millions of sequential neural network passes, making decoding prohibitively slow (taking minutes or hours). The primary challenge addressed by L3C is leveraging the predictive power of deep learning for lossless compression while overcoming the computational bottleneck of pixel-by-pixel processing.

Methods

The proposed L3C system utilizes a hierarchical probabilistic model that replaces sequential autoregression with a few parallel passes. The architecture is centered around three key components:

  1. Learned Auxiliary Representations: Instead of modeling RGB pixels directly, the system learns a hierarchy of latent features ($z^{(1)}, z^{(2)}, z^{(3)}$).
  2. Parallel Predictors: The model employs a series of feature extractors ($E$) and predictors ($D$). A predictor at a coarse scale predicts the probability distribution of features at the next, finer scale. Since these predictions are made for all pixels in a layer simultaneously, the model is fully parallelizable on a GPU.
  3. Soft Quantization: To enable end-to-end training, the authors use a differentiable "soft" quantization during training to approximate the hard rounding required for actual compression.
The L3C hierarchical architecture showing the flow from input image $x$ through encoders $E$ to latent representations $z$, and the subsequent decoding via predictors $D$ to obtain pixel probabilities.

As shown in Figure 1, the model processes the image across multiple scales. The authors found that learning these hierarchical representations is critical; using a simple "RGB pyramid" (downscaled versions of the image) performed significantly worse than allowing the network to discover its own features.

Detailed internal structure of the encoder $E^{(s)}$ and decoder $D^{(s)}$ blocks, highlighting the use of residual connections and the prediction of distribution parameters ($\pi, \mu, \sigma$).

The internal blocks (Figure 2) are designed for efficiency and expressiveness, utilizing residual layers to capture complex dependencies. The predicted probabilities are eventually fed into an Adaptive Arithmetic Coder (AAC) to produce the final bitstream.

Results

L3C was evaluated on high-standard datasets including Open Images, DIV2K (2K resolution), and RAISE-1k (raw images).

  • Compression Performance: L3C consistently outperforms PNG (by approximately 40%), JPEG 2000, and WebP across all tested datasets.
  • Computational Efficiency: On a $320 \times 320$ image, L3C is over 50,000 times faster than PixelCNN++ and 500 times faster than MS-PixelCNN. It can decode a $512 \times 512$ image in roughly 0.37 seconds on a modern GPU.
  • Comparison to State-of-the-Art: While L3C surpasses most classical codecs, it is still marginally outperformed (by 4-13%) by FLIF, a highly specialized, hand-engineered non-learned codec.

Discussion

L3C represents a significant step toward practical learned lossless compression. However, some limitations remain. Currently, the system requires a GPU for competitive performance, whereas classical codecs are highly optimized for CPUs. Additionally, there is still a small performance gap compared to FLIF.

Future work may involve exploring "weak autoregression"—combining the parallel approach with small local autoregressive windows—or adapting the model dynamically to specific images to further improve bitrates. The authors also highlight potential applications in specialized domains like medical imaging, where lossless compression is often a requirement.

Summary updated

Abstract

We propose the first practical learned lossless image compression system, L3C, and show that it outperforms the popular engineered codecs, PNG, WebP and JPEG 2000. At the core of our method is a fully parallelizable hierarchical probabilistic model for adaptive entropy coding which is optimized end-to-end for the compression task. In contrast to recent autoregressive discrete probabilistic models such as PixelCNN, our method i) models the image distribution jointly with learned auxiliary representations instead of exclusively modeling the image distribution in RGB space, and ii) only requires three forward-passes to predict all pixel probabilities instead of one for each pixel. As a result, L3C obtains over two orders of magnitude speedups when sampling compared to the fastest PixelCNN variant (Multiscale-PixelCNN). Furthermore, we find that learning the auxiliary representation is crucial and outperforms predefined auxiliary representations such as an RGB pyramid significantly.
BibTeX
@inproceedings{mentzer2019practical,
  author = {Fabian Mentzer and Eirikur Agustsson and Michael Tschannen and Radu Timofte and Luc Van Gool},
  title = {{Practical Full Resolution Learned Lossless Image Compression}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  url = {https://openreview.net/forum?id=1QUnJYfSAX},
}
Figure