Abstract
The leading approach for image compression with artificial neural networks (ANNs) is to learn a nonlinear transform and a fixed entropy model that are optimized for rate-distortion performance. We show that this approach can be significantly improved by incorporating spatially local, image-dependent entropy models. The key insight is that existing ANN-based methods learn an entropy model that is shared between the encoder and decoder, but they do not transmit any side information that would allow the model to adapt to the structure of a specific image. We present a method for augmenting ANN-based image coders with image-dependent side information that leads to a 17.8% rate reduction over a state-of-the-art ANN-based baseline model on a standard evaluation set, and 70-98% reductions on images with low visual complexity that are poorly captured by a fixed, global entropy model.
Introduction
In the field of learned image compression, Artificial Neural Networks (ANNs) are typically used to transform an image into a set of discrete, quantized codes. To achieve high compression ratios, these codes must be entropy coded using a probability model. Traditionally, these models have been "fixed" and "global"—meaning a single probability distribution, learned during training on millions of images, is applied to every part of every new image processed at test time.
However, this "one-size-fits-all" approach is inherently inefficient. Natural images exhibit vast local statistical variation; a clear sky has very different characteristics than a dense forest or a city skyline. A fixed model must compromise by assuming an average complexity, which leads to significant bit-rate waste when processing regions that are simpler or more structured than the average. This paper introduces a method to overcome this limitation by allowing the entropy model to adapt to the specific local structure of an image.
Methods: The SLIMD Framework
The proposed solution is SLIMD (Spatially Local, Image-dependent Multinomial Dictionary). Unlike standard neural codecs, SLIMD introduces forward-adaptation by transmitting a small amount of side information to the decoder, enabling it to use a probability model tailored to the image at hand.
As shown in Figure 1, the system operates on three core principles:
- Local Modeling: The quantized code tensor is divided into spatial tiles. Instead of applying one global model, the system evaluates statistics locally for each tile.
- Multinomial Dictionary: The system utilizes a pre-learned dictionary of 256 distinct probability distributions (multinomials). This dictionary is learned offline using a modified K-means++ algorithm designed to minimize KL-divergence across a diverse training set.
- Adaptive Selection: For every tile, the encoder identifies the dictionary entry that minimizes the cross-entropy (and thus the bit-rate). The index of this chosen distribution is sent as side information. If no dictionary entry fits well, the encoder can even transmit a custom, image-specific distribution if the resulting bit savings justify the overhead.
Key Results and Evaluation
The SLIMD model was evaluated against a state-of-the-art ANN-based baseline using the Tecnick (100 high-resolution images) and Kodak PhotoCD datasets.
Performance Gains
On the Tecnick dataset, SLIMD achieved an average 17.88% bit-rate reduction over the baseline when optimized for Mean Squared Error (MSE). When optimized for perceptual quality (MS-SSIM), it maintained a strong lead with a 12.98% reduction. These gains make neural compression highly competitive with top-tier traditional codecs like BPG (Better Portable Graphics).
Handling Low Complexity
The most dramatic improvements were observed in images with low visual complexity. Fixed models often "over-prepare" for complexity that isn't present in simple images, such as solid colors or smooth gradients.
As illustrated in Figure 2, while the average saving is nearly 18%, specific images saw reductions as high as 98.5%. By allowing the entropy model to collapse onto the actual local distribution, SLIMD effectively eliminates the "complexity tax" imposed by fixed global models.
Conclusion
This research demonstrates that the perceived "cost" of sending side information in neural compression is a worthwhile investment. By spending a few bits to describe the local statistical landscape of an image, SLIMD achieves substantial overall bit-rate savings. This shift from static to adaptive modeling represents a significant step forward in making learned image compression both more efficient and more robust across diverse image types.
Summary updated