Abstract
Precise estimation of the probabilistic structure of natural images plays an essential role in image compression. Despite the recent remarkable success of end-to-end optimized image compression, the latent codes are usually assumed to be fully statistically factorized in order to simplify entropy modeling. However, this assumption generally does not hold true and may hinder compression performance. Here we present context-based convolutional networks (CCNs) for efficient and effective entropy modeling. In particular, a 3D zigzag scanning order and a 3D code dividing technique are introduced to define proper coding contexts for parallel entropy decoding, both of which boil down to place translation-invariant binary masks on convolution filters of CCNs. We demonstrate the promise of CCNs for entropy modeling in both lossless and lossy image compression. Experiments on the Kodak and Tecnick datasets show that our methods powered by the proposed CCNs generally achieve comparable compression performance to the state-of-the-art while being much faster.
Introduction
In image compression, entropy modeling aims to predict the probability distribution of data to minimize the bit rate. While modern deep-learning compressors often assume latent codes are statistically independent (factorized models) for speed, this approach ignores crucial spatial relationships. Context-based models like PixelCNN improve accuracy by considering neighboring pixels but suffer from slow, sequential decoding. This work introduces Context-based Convolutional Networks (CCNs) to bridge the gap between high accuracy and real-world decoding speeds.
Context-based Convolutional Networks (CCNs)
The core innovation of CCNs lies in their ability to maintain context awareness while enabling parallelization. This is achieved through two primary techniques:
- 3D Zigzag Scanning and Masking: To ensure the model only utilizes previously decoded information, translation-invariant binary masks are applied to the convolution filters. This is extended to 3D to accommodate multi-channel latent representations.
- 3D Code Dividing: This technique enables parallel decoding by partitioning data into groups (diagonal planes in 3D space). Within each group, pixels are conditionally independent given the preceding groups, allowing for simultaneous decoding on a GPU.
Entropy Modeling Pipelines
The CCN framework is applied to both lossless and lossy compression tasks:
- Lossless Compression: The image is represented as bit-planes, and the CCN predicts a Bernoulli distribution for each bit to estimate entropy.
- Lossy Compression: The model employs a Discretized Mixture of Gaussian (MoG) distributions. By modeling data as a combination of multiple Gaussians, the CCN captures the complex, multimodal structures of natural images more effectively than single-Gaussian models.
Experimental Results
The models were trained on 10,000 high-quality Flickr images and evaluated on the Kodak and Tecnick benchmarks.
- Performance: In lossless compression, CCN achieved bit savings of 5.9% to 6.2% over standards like JPEG2000-LS and BPG. In lossy compression, it matched or exceeded state-of-the-art DNN-based compressors.
- Efficiency: The most significant result is the speedup; CCN is approximately 30 times faster at decoding than previous context-based models like MCN. A "CCNlight" variant offers even greater speeds with minimal accuracy loss.
Conclusion
CCNs successfully address the speed bottleneck of context-aware entropy modeling. While performance plateaus at very high bit rates due to limited model capacity, the framework shows great promise. Future work will explore scaling the architecture and applying CCNs to other generative modeling tasks.
Summary updated