In traditional image compression like JPEG, the math is standardized and simple. But in neural data compression, we face a "catastrophic agreement" problem. If a neural network on an NVIDIA GPU calculates a probability as $0.7000001$ and a mobile phone CPU calculates it as $0.7000002$, the entire decoding process fails.
This sensitivity arises because modern compression relies on entropy coding, a mechanism that requires the encoder and decoder to maintain perfectly synchronized probability distributions to map bits to data. Even a tiny floating-point rounding error—common when moving across different hardware architectures—causes the decoder to lose its place in the bitstream, leading to garbled output or total crashes.
The Fragility of Entropy Coding
To understand why this happens, we must look at how Nonlinear Transform Coding (NTC) works. As Jona Ballé and Johannes Ballé describe, neural codecs transform pixels into a latent space, quantize them, and then use a "hyperprior" network to predict the probability distribution of those latents.
The arithmetic coder uses these predicted probabilities to pack the data into the shortest possible bitstream. If the decoder’s hardware (e.g., a mobile ARM processor) produces a slightly different probability than the encoder’s hardware (e.g., a server-grade GPU), the arithmetic decoder will read the wrong number of bits for a symbol. This "desynchronization" is cumulative; once one symbol is misread, every subsequent symbol in the file is interpreted incorrectly.
Solving the Numerical Drift
Researchers have proposed three primary strategies to ensure that neural codecs can work reliably across heterogeneous hardware.
1. Integerization and Fixed-Point Arithmetic
The most direct solution is to abandon floating-point math entirely during the inference phase of the decoder. Jona Ballé and colleagues introduced Integer Networks in Integer Networks for Data Compression with Latent-Variable Models. By using deterministic integer arithmetic, they ensure that every device, regardless of its underlying architecture, arrives at the exact same bit-perfect result. Similarly, Junqi Shi and Ming Lu developed RDO-PTQ, which converts heavy floating-point models into 8-bit fixed-point versions, maintaining compression efficiency while ensuring cross-platform consistency.
2. Codebook-Based Indices
Instead of having the network predict a continuous probability distribution, some researchers suggest transmitting discrete indices. Jinxi Xiang and Xiao Han proposed a codebook-based method for video coding. By transmitting integer indices from a shared codebook, the system bypasses the need for the decoder to perform complex, non-deterministic probability estimations. This ensures that the "meaning" of the bitstream remains identical across an NVIDIA V100 and a mobile phone.
3. Mismatch-Tolerant Coding
A more recent approach acknowledges that some numerical drift might be inevitable and seeks to make the coding process robust to it. Aviv Adler and Jennifer Tang introduced PMATIC (Probability-Mismatch-Aware Transformation for Information Coding).
In Synchronizing Probabilities in Model-Driven Lossless Compression, they describe a system that uses a small amount of "helper bits" to synchronize quantized probabilities between the encoder and decoder. This allows Large Language Models (LLMs) to perform lossless compression even if the hardware produces slightly different logits, providing a 10–15x increase in robustness over standard methods.
Implications for Deployment and Safety
The need for determinism extends beyond simple image viewing. In the realm of AI safety and steganography, Antonio Norelli and Michael M. Bronstein demonstrate with Calgacus that hiding secret messages within LLM-generated text relies on exact probability rankings. If the sender and receiver use different GPUs, the hidden message can be completely corrupted.
As neural codecs move toward standardization—aiming to outperform classical standards like HEVC and VVC—achieving bit-perfect cross-platform determinism remains the final hurdle for real-world interoperability. Whether through strict integerization or robust error-correcting codes, the goal is the same: ensuring that the "agreement" between encoder and decoder never breaks.