Optimal Neural Compressors for the Rate-Distortion-Perception Tradeoff

Eric Lei, Hamed Hassani, Shirin Saeedi Bidokhti

Abstract

Traditional data compression focuses on the tradeoff between bit rate and reconstruction error (distortion). However, modern applications increasingly prioritize perceptual quality—how realistic an image or signal appears to a human observer. While information theory suggests that achieving the optimal Rate-Distortion-Perception (RDP) tradeoff requires shared randomness between the encoder and decoder, practical implementations have remained elusive. This paper introduces a family of low-complexity neural compressors based on Lattice Transform Coding (LTC) and dithered quantization. By varying the amount of shared randomness, these models achieve theoretical optimality limits, bridging the gap between abstract information theory and constructive neural network design.

Introduction: The RDP Tradeoff

In the field of data compression, the relationship between Rate (the number of bits used) and Distortion (the error between original and reconstruction) is well-understood. Recently, a third dimension has gained prominence: Perception. Perceptual quality is defined by how closely the distribution of the reconstructed data matches the distribution of the source data. A perfectly "perceptual" reconstruction is indistinguishable from the original to a discriminator, even if it is not a pixel-perfect match.

Information theory dictates that to achieve the mathematically optimal RDP tradeoff, a compressor must utilize shared randomness between the encoder and decoder. Previous theoretical proofs relied on highly complex schemes like Reverse Channel Coding (RCC), which are computationally prohibitive for real-world neural networks. Existing neural compressors often ignore this requirement, use ad-hoc noise injection, or suffer from high computational costs. This research addresses the gap by designing practical, low-complexity neural compressors that allow for precise control over shared randomness.

Methodology: Lattice Transform Coding

The authors build their framework on Lattice Transform Coding (LTC). While standard neural compressors typically use scalar quantization (rounding latents to the nearest integer), LTC employs Lattice Quantization (LQ). Lattices provide a more efficient way to pack multi-dimensional space—similar to how hexagonal honeycombs pack 2D space more efficiently than a square grid—thereby reducing distortion for a given rate.

To satisfy the perception constraint, the compressor must be randomized. The authors achieve this by introducing a random dither (noise) over the lattice cells. They propose three distinct architectures based on the availability of shared randomness:

  1. SD-LTC (Shared-Dither): Assumes infinite shared randomness. A continuous random dither is subtracted before quantization at the encoder and added back at the decoder. This allows the quantization error to behave like a clean convolution, which is mathematically ideal for RDP.
  2. PD-LTC (Private-Dither): Assumes zero shared randomness. The encoder operates deterministically, and the dither is only added at the decoder to ensure the output distribution matches the source. While less efficient than SD-LTC, it is easier to implement in systems without shared seeds.
  3. QSD-LTC (Quantized Shared-Dither): Assumes finite shared randomness. Using a "nested lattice" structure, this model shares a discrete, quantized dither between the encoder and decoder, effectively interpolating between the private and infinite shared randomness extremes.

Experimental Results

The models were evaluated on a variety of datasets, including synthetic 8-dimensional Gaussian sources (to verify theoretical bounds), MNIST images, 16-dimensional physics measurements, and 33-dimensional speech signals.

Matching Theoretical Limits

On Gaussian sources, the SD-LTC model perfectly matched the theoretical RDP lower bound. It achieved the same performance as the complex RCC method but at a fraction of the computational cost. This confirms that lattice-based dithering is a constructive way to reach the fundamental limits of information theory.

The Impact of Lattice Efficiency

Across all datasets, the choice of lattice structure proved critical. Models utilizing more efficient lattices (such as the $E_8$ or $\Lambda_{16}$ lattices) consistently yielded better RDP curves than those using standard integer grids. This demonstrates that the geometric efficiency of the quantizer directly translates to better compression performance.

The Necessity of Randomness

A key finding was that deterministic models completely failed to enforce perception constraints at low bitrates. Regardless of how much weight was given to the perception loss during training, deterministic encoders could not produce realistic reconstructions without the injection of dither. Furthermore, the QSD-LTC model showed that "infinite" randomness is not required for practical gains; sharing as little as 1.58 bits per dimension (a nesting ratio of 3) allowed the model to nearly match the performance of the infinite-randomness SD-LTC.

Conclusion and Future Work

This work provides a practical blueprint for designing neural compressors that are optimized for the Rate-Distortion-Perception tradeoff. By combining the packing efficiency of lattices with the distributional benefits of dithering, the authors have created a low-complexity solution that meets theoretical benchmarks.

Future research will focus on expanding the range of achievable randomness rates for the QSD-LTC model and integrating these dithered lattice techniques into state-of-the-art image compression architectures. A significant challenge remains in merging these techniques with complex hyperprior models, which currently rely on deterministic assumptions.

Summary updated

BibTeX
@inproceedings{lei2025optimal,
  author = {Eric Lei and Hamed Hassani and Shirin Saeedi Bidokhti},
  title = {{Optimal Neural Compressors for the Rate-Distortion-Perception Tradeoff}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=ZAKpELpclI},
}
Figure