Compression as a Denoising Regularizer

This direction is split into other directions: compression-based denoising. Its last version stays readable and citable here.

Version 1 of 146 papers69 conceptsfrozen 2026-10-04

In traditional signal processing, compression and denoising are often viewed as sequential, competing steps. You denoise an image to make it look better, then you compress it to save space. However, a growing body of research suggests that these two tasks are fundamentally the same operation viewed through different lenses. By treating compression as a form of regularization, researchers have found that the "information bottleneck" inherent in lossy compression can be the most effective tool for purifying a signal.

The Incompressibility of Noise

The core intuition behind this research direction is an information-theoretic one: structured signals are compressible, while random noise is not.

In any data source, whether it is an infrared image from a satellite or a genomic sequence, the "signal" consists of patterns, symmetries, and redundancies. These can be described with a small number of bits. "Noise," by definition, is stochastic and lacks such patterns. As Ayfer Ozgur and Tsachy Weissman demonstrate in The Performance of Compression-Based Denoisers, if you calibrate a compressor's distortion level to the entropy of the noise, the compressor will naturally discard the high-entropy noise while preserving the low-entropy signal.

This principle was applied as early as 2005 by Ahmed Nabil Belbachir and Roland Ottensamer in the Herschel-PACS camera pipeline. They realized that in high-noise environments, standard lossless algorithms fail because they try to faithfully represent every random fluctuation. By extracting the underlying physical signal on-board—effectively a lossy reduction—they could achieve massive telemetry savings without losing scientific data.

The Bit Misallocation Problem

When compression and denoising are handled separately, a "bit misallocation" problem occurs. Standard codecs like JPEG or vanilla 3D Gaussian Splatting treat every pixel or point as equally important. If an image is noisy, the encoder wastefully spends its limited bit budget trying to reconstruct that noise.

Ka Leong Cheng and Qifeng Chen address this in Optimizing Image Compression via Joint Learning with Denoising. They argue that by training a compressor and a denoiser together, the model learns to ignore sensor noise at the feature level. Similarly, Jin, Jian shows in Make Lossy Compression Meaningful for Low-Light Images that enhancing low-light images before compression is a mistake; it amplifies noise that the compressor then struggles to encode. The solution is an integrated architecture where the compression bottleneck acts as a gatekeeper, ensuring only meaningful visual information consumes bandwidth.

Latent Space Separation

A powerful mechanism for this purification is the use of "scalable" or "nested" latent spaces. Instead of just discarding data, these models segregate it.

In Joint image compression and denoising via latent-space scalability, Saeed Ranjbar Alvar and Ivan V. Bajic describe a system that splits data into a base layer (the clean signal) and an enhancement layer (the noise). Because noise is high-entropy, it is naturally relegated to the secondary layer when the primary layer is under a strict bitrate constraint.

This concept extends to deep learning regularization. Yingyi Chen and Shell Xu Hu found that Nested Dropout—a technique that forces information into a limited number of dimensions—acts as a filter for label noise. By forcing the network to compress its knowledge, it prioritizes the structured signal of the correct labels over the unstructured "noise" of incorrect ones.

Zero-Shot Denoising through Compression

Perhaps the most striking evidence for this idea is that you can denoise an image without ever seeing a "clean" example. Ali Zafari and Xi Chen introduced ZS-NCD, which denoises by simply attempting to compress patches of a single noisy image.

Because the neural network is optimized for a rate-distortion objective, it finds it "cheaper" to learn the recurring structures of the scene than to learn the unique, high-entropy noise of each pixel. This turns compression into a "universal prior." As Zhihao Li and Bihan Wen demonstrate in Compress Clean Signal from Noisy Raw Image, this approach can reduce file sizes by up to 50x while simultaneously improving image quality by 10dB.

Beyond Visual Data

The principle that "compression equals purification" is not limited to pixels. It is being applied to:

  • 3D Scenes: Lanqing Guo uses entropy constraints in ContextGS to filter out noise in 3D Gaussian Splatting, leading to cleaner renders than uncompressed models.
  • Medical Imaging: Veronica Corona shows that integrating lossy compression into MRI reconstruction acts as a complexity regularizer, preventing the model from fitting acquisition artifacts.
  • Language Models: Jinheon Baek developed CoLoR, which compresses long-context inputs for LLMs. By removing irrelevant "noise" from the text, the model's retrieval accuracy actually increases by 6% even as the input size is halved.

In all these cases, the "loss" in lossy compression is not a bug, but a feature. By intentionally discarding information that is too complex to describe efficiently, these systems filter out the chaos of the real world to reveal the underlying signal.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_compression_as_a_denoising_regularizer_a669ea9a,
  title = {Compression as a Denoising Regularizer},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compression-as-a-denoising-regularizer/rdr_a669ea9a532e49ac956221f84d24fae1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_compression_as_a_denoising_regularizer_a669ea9a_v1,
  title = {Compression as a Denoising Regularizer},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compression-as-a-denoising-regularizer/rdr_a669ea9a532e49ac956221f84d24fae1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}