Data compression has fundamentally shifted from handcrafted, deterministic algorithms to learned, probabilistic models. By framing compression as a density estimation problem, neural networks can capture intricate, high-dimensional patterns that traditional codecs miss entirely. For instance, Dileep Kalathil et al. demonstrated in LLMZip: Lossless Text Compression using Large Language Models that large language models can push the boundaries of text compression to unprecedented limits. Similarly, in biological domains, the authors of Implicit neural image field for biological microscopy image compression achieve massive compression ratios by treating data as continuous neural fields.
Yet, a severe bottleneck prevents these models from replacing standard tools like gzip or JPEG: computational latency. As early as 1994, Jürgen Schmidhuber et al. noted in Predictive Coding with Neural Nets: Application to Text Compression that neural prediction models could be up to three orders of magnitude slower than traditional dictionary-based algorithms. Decades later, this fundamental trade-off persists. To make neural compression practical, researchers are actively diagnosing why these networks are slow and designing architectures to break through the latency bottleneck.
The Inference Bottleneck: Autoregression and Heavy Decoders
Why does decoding a neural bitstream take so long? The primary culprit is often the sequential nature of state-of-the-art entropy models. In lossless compression, to decode the next token or pixel, the model must know the exact probability distribution conditioned on all previously decoded elements.
As David Heurtel-Depeiges et al. observe in Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data, token-by-token autoregressive decoding forces the hardware to execute massive neural network forward passes sequentially. This completely neutralizes the parallel processing power of modern GPUs.
Even when moving away from large language models to specialized geometric structures, the computational overhead remains high. For example, the tree-structured entropy models analyzed by Lila Huang et al. in OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression require multiple multi-layer perceptron passes. Because these passes must occur step-by-step down the data structure, they introduce significant computational latency that hinders real-time deployment on vehicle hardware.
The Encoding Bottleneck: Per-Instance Optimization
While inference latency plagues the receiver, an entirely different bottleneck occurs at the sender's side, particularly in Implicit Neural Representations (INRs). Instead of passing data through a feed-forward encoder, INR-based codecs compress data by overfitting a unique neural network to each individual data instance. The network's weights themselves become the compressed representation.
The authors of RECOMBINER: Robust and Enhanced Compression with Bayesian Implicit Neural Representations point out that this paradigm requires an expensive iterative optimization process via gradient descent. As highlighted by Rishit Dagli et al. in Squeeze3D: Your 3D Generation Model is Secretly an Extreme Neural Compressor, this per-object optimization can take minutes or even hours for a single asset. This training overhead makes such methods less suitable for real-time streaming or write-intensive workloads.
Accelerating the Pipeline: Restructuring and Disentanglement
To bridge the gap between high-fidelity neural compression and real-time performance, researchers are attacking the latency problem from multiple angles: architectural restructuring, auxiliary transforms, and lightweight feature modeling.
Causal Context Adjustment and Grouping
One way to mitigate decoding latency is to optimize how the model processes context. Minghao Han et al. introduce a novel training objective in Causal Context Adjustment Loss for Learned Image Compression. By explicitly guiding the model to prioritize the most informative data during the early stages of decoding and pairing it with an unevenly grouped convolutional structure, they managed to slash inference latency by over 20% while maintaining state-of-the-art rate-distortion performance.
Disentangled Training
Training neural codecs is notoriously slow because the network must simultaneously learn coarse energy compaction and fine-grained detail reconstruction. In On Disentangled Training for Nonlinear Transform in Learned Image Compression, Han Li proposes a lightweight, linear auxiliary transform called AuxT. By offloading the coarse energy compaction to this simple linear layer, the main nonlinear neural network can focus entirely on fine details, accelerating training speed by 2x.
Traditional Pipeline:
[Raw Data] ---> [ Heavy Nonlinear Neural Network ] ---> [Compressed Bits]
(Must learn both coarse & fine details)
Disentangled Pipeline (AuxT):
---> [ Lightweight Linear AuxT ] ----> (Coarse Compaction)
[Raw Data] --| |---> [Compressed Bits]
---> [ Main Nonlinear Network ] ------> (Fine Details Only)
Lightweight Feature Models and Matrix Decomposition
Rather than relying on heavy recurrent architectures or deep autoencoders, some approaches substitute complex neural operations with classical mathematical decompositions guided by lightweight models.
Jihoon Ko introduces ELiCiT in Effective and lightweight lossy compression of tensors: techniques and applications, which replaces autoregressive networks with a lightweight feature-based index model and clustering-based quantization, yielding up to 96× faster compression than deep learning baselines. Similarly, Yu-Ting Zhan et al. present CAT-3DGS Pro: A New Benchmark for Efficient 3DGS Compression, which leverages a PCA-guided vector-matrix hyperprior to bypass complex triplane structures, achieving a 5× speedup in decoding.
The evolution of neural compression is no longer just a race for the lowest bitrate; it is a race for the lowest latency. By shifting from heavy, fully autoregressive models to hybrid architectures that combine lightweight neural priors with parallelizable classical transforms, the field is steadily moving toward real-time viability.
Key Foundational Work
Predictive Coding with Neural Nets: Application to Text Compression (1994) — contributed 2 concept elements to this direction.