Deep learning has revolutionized data compression. By replacing traditional linear transforms with deep neural networks, modern codecs can compress images, video, and language model representations far more efficiently than classical standards. However, deploying these models in production reveals a hidden, catastrophic vulnerability: floating-point non-determinism.
When a neural network runs on heterogeneous hardware—such as encoding on a cloud GPU and decoding on a consumer CPU—subtle differences in how processors handle floating-point arithmetic cause the system to break. Understanding why this happens, and how researchers are fixing it, is essential for bringing neural compression out of the lab and into the real world.
The Fragility of Entropy Coding
To understand why floating-point variations are fatal, we must look at the final step of any compression pipeline: entropy coding.
Neural compression models typically output continuous latent representations, which are quantized and then mapped to a compact binary bitstream using algorithms like arithmetic coding. Arithmetic coding relies on a shared mathematical agreement: the encoder and the decoder must calculate the exact same probability distribution for the next symbol. The algorithm uses these probabilities to carve out precise, nested numerical intervals between 0 and 1.
Encoder (GPU): Calculates P(symbol) = 0.70000001 -> Allocates Interval A
Decoder (CPU): Calculates P(symbol) = 0.70000000 -> Allocates Interval B (Shifted!)
Result: Bitstream desynchronizes completely; decoding fails.
As Ballé et al. and Ballé et al. demonstrate, entropy decoding is highly sensitive to minor numerical discrepancies. If a GPU encoder estimates a probability as 0.70000001 and a CPU decoder computes it as 0.70000000, their interval boundaries diverge. A single mismatched bit causes the decoder to misinterpret every subsequent symbol, leading to total desynchronization and catastrophic decoding failure.
This requirement for bit-exact consistency poses a massive challenge across heterogeneous computing environments, as noted by Wu et al. and Lester et al..
The Compounding Effect of Temporal Drift
The problem worsens dramatically in sequential or predictive architectures, such as video codecs or Large Language Models (LLMs).
In neural video compression, frames are decoded sequentially using a temporal prediction loop: the reconstruction of Frame $N$ serves as the reference baseline for predicting Frame $N+1$. As explored by Conceição et al. and Song et al., minor rounding errors introduced by hardware-dependent floating-point math do not remain isolated. Instead, they propagate and amplify frame-by-frame. This numerical "drift" quickly corrupts the internal state of the model, resulting in severe visual degradation that eventually renders the video unwatchable.
This sensitivity is not unique to video. He et al. highlight that compression schemes utilizing precise mathematical transformations for LLM weights are similarly fragile; small numerical drifts during sequential decoding require explicit error-correction mechanisms to maintain model integrity.
Why Floating-Point Math Diverges
Why do different processors produce different answers for the same network weights? The IEEE 754 standard defines floating-point behavior, but it allows hardware vendors flexibility in implementation details, such as:
- Fused Multiply-Add (FMA) operations: Combining a multiplication and an addition into a single step with one rounding operation versus performing them separately.
- Library-level optimizations: Variations in parallel reduction orders or transcendental function approximations across software libraries (e.g., CUDA vs. OpenCL).
As Brand et al. and Jiahao Li et al. observe, these hardware-induced variances mean that a compressed bitstream generated on one device cannot be reliably decoded on another. Even in image restoration tasks, Ohayon et al. point out that floating-point discrepancies between software implementations prevent models from satisfying strict mathematical consistency constraints across environments. Similarly, Magli notes that variations between CPU and GPU rounding results prevent identical reconstruction in onboard satellite hyperspectral imaging unless extra synchronization data is introduced.
To overcome this bottleneck, researchers have developed three primary strategies: strict integerization, mismatch-tolerant coding, and entropy-free architectures.
Strategy 1: Strict Integerization and Fixed-Point Math
The most direct way to achieve cross-platform determinism is to banish floating-point operations entirely from the deployment path. By forcing the network to operate solely on integer math, every processor is guaranteed to produce identical bit-level outputs.
- Integer Networks: Ballé et al. introduced a framework for training neural networks using deterministic integer arithmetic, ensuring bit-perfect agreement between encoders and decoders regardless of the underlying hardware.
- Fixed-Point Transforms: In classical and hybrid video standards, high-precision linear transforms are converted into fixed-point integer representations to eliminate encoder-decoder synchronization drift (Fan et al.).
- Integer Inference for Video: To stabilize recursive video pipelines, Bin Li et al. implemented 16-bit integer operations for neural network inference, preventing the accumulation of temporal drift.
- Targeted Quantization: Recognizing that the hyperprior path is the most vulnerable to desynchronization, Conceição et al. proposed a post-training static 8-bit quantization specifically for the hyperprior decoding path, securing cross-platform determinism with minimal impact on compression efficiency.
Strategy 2: Mismatch-Tolerant Coding and Calibration
Enforcing strict integer operations can sometimes degrade compression performance or require extensive model retraining. An alternative approach allows the underlying hardware to remain non-deterministic but modifies the coding layer to handle the resulting variance.
Adler et al. developed an optimized interval coding framework called PMATIC. Instead of assuming the encoder and decoder share identical probability distributions, PMATIC builds a mathematical buffer into the arithmetic coding intervals. This design allows the framework to tolerate slight prediction mismatches caused by hardware non-determinism, achieving robust lossless compression across heterogeneous platforms without sacrificing efficiency.
Alternatively, models can transmit explicit correction data to realign the decoder. Tian et al. introduced a "calibration information transmitting" strategy for neural video codecs. By calculating the drift on the encoder side and sending targeted calibration data within the bitstream, the decoder can correct its internal states dynamically without needing strict integer-only constraints.
Strategy 3: Bypassing Entropy Coding Entirely
If variable-length entropy coding is the component that breaks under floating-point variance, a radical but effective solution is to eliminate it altogether.
Traditional Neural Compression:
Latents -> Quantization -> Floating-Point Probabilities -> Variable-Length Coding (Fragile)
Entropy-Free Compression (EF-LIC):
Latents -> Unconstrained Vector Quantization -> Fixed-Length Indices (Robust)
Cao et al. introduced EF-LIC (Entropy-Coding Free Learned Image Compression). Instead of mapping quantized latents to variable-length bitstreams based on continuous probability estimations, EF-LIC utilizes unconstrained vector quantization to output fixed-length indices. Because fixed-length indices do not rely on dynamically computed probabilities to establish bitstream boundaries, a minor floating-point discrepancy in the network's internal layers cannot cause desynchronization. This architectural shift inherently protects the system from cross-platform numerical drift while significantly accelerating decoding speeds.
Key Foundational Work
Integer Networks for Data Compression with Latent-Variable Models (2019) — contributed 2 concept elements to this direction.
A practical convolutional neural network as loop filter for intra frame (2018) — contributed 1 concept elements to this direction.
Signal-Independent Separable KLT by Offline Training for Video Coding (2019) — contributed 1 concept elements to this direction.