Neural Artifact Reduction for Compressed Video

This direction is split into other directions: video compression artifacts. Its last version stays readable and citable here.

Version 1 of 175 papers79 conceptsfrozen 2026-10-04

Modern video streaming is a constant battle between visual quality and bandwidth. To fit high-definition video through narrow network pipes, codecs like HEVC and VVC must aggressively discard data. This process, known as lossy compression, introduces a predictable set of visual "scars": blocky edges, blurry textures, and "ringing" halos around sharp objects.

While traditional codecs use hand-crafted filters to smooth these errors, a new generation of research—often termed Compressed Video Quality Enhancement (CVQE)—replaces these heuristics with deep learning. By treating compression artifacts as a structured denoising problem, researchers are finding ways to restore lost details and significantly reduce the data needed for a high-quality experience.

The Anatomy of a Compression Artifact

To understand why neural networks are effective here, we must first understand why artifacts appear. Most video codecs use block-based transform coding. They divide a frame into a grid (blocks) and apply a mathematical transform to turn pixels into frequencies.

The "lossy" part happens during quantization, where high-frequency details (like the fine texture of hair or skin) are rounded off to zero to save space. As Jingxuan Hou and Fan Zhang note, this quantization creates discontinuities at the edges of blocks, leading to the familiar "grid" look in low-quality video Quality Enhancement of Compressed Video via CNNs.

Traditional filters, like the deblocking filters in H.264, are simple and linear. They struggle to distinguish between a real edge in the scene and an artificial edge created by the codec. Neural networks, however, can learn the complex "fingerprint" of compression noise, allowing them to suppress artifacts while preserving—or even hallucinating—original textures.

Post-Processing vs In-Loop Filtering

Research in this area generally follows two paths: fixing the video after it is decoded (post-processing) or fixing it while it is being decoded (in-loop filtering).

Post-Processing: The Cleanup Crew

Post-processing models like DIA_Net Post-Processing Network Based on Dense Inception Attention for Video Compression act as a final "polish" layer. These models are attractive because they don't require changing the underlying video standard; they can be applied to any existing stream.

The challenge in post-processing is temporal consistency. If a model cleans each frame differently, the video will appear to "flicker." To solve this, models like NL-ConvLSTM Non-Local ConvLSTM for Video Compression Artifact Reduction by Yi Xu look at neighboring frames to find high-quality details that might have survived in one frame but were lost in another.

In-Loop Filtering: The Internal Fix

In-loop filters are integrated directly into the codec's prediction loop. When a codec predicts what the next frame will look like, it uses the "cleaned up" version of the previous frame. This creates a virtuous cycle: better filtering leads to better predictions, which leads to even lower bitrates.

Architectures like MFRNet MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering and RRNet Residual-Guided In-Loop Filter Using Convolution Neural Network have shown that by reusing high-dimensional features across the codec, they can achieve bitrate savings of over 20% compared to standard HEVC.

Leveraging Hidden Metadata

One of the most powerful insights in recent work is that the video bitstream contains "free" hints that can guide restoration. When a video is compressed, the encoder makes decisions about motion and quality that are usually discarded after decoding.

Models like MetaBit Leveraging Bitstream Metadata for Fast and Accurate Video Compression Correction and CPGA CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement use this metadata—specifically motion vectors and quantization parameters (QP)—as inputs.

  • Motion Vectors tell the model exactly how objects moved between frames, replacing the need for expensive and error-prone optical flow estimation.
  • QP values tell the model exactly how much "noise" to expect in a specific block, allowing a single neural network to adapt its strength dynamically A QP-adaptive Mechanism for CNN-based Filter in Video Coding.

From Pixels to Semantics: Generative Coding

At ultra-low bitrates (e.g., 100 bits per second), traditional pixel-based reconstruction fails entirely. No amount of filtering can recover a face from a handful of blocks. This has led to a shift toward Generative Face Video Coding (GFVC).

Instead of trying to compress pixels, researchers like Shanzhi Yin and Bolin Chen propose compressing "intent." In Dynamics-Codec Compressing Scene Dynamics: A Generative Approach, the system converts facial movements into compact mathematical tokens. The receiver then uses a generative model (like a GAN or Diffusion model) to "re-animate" the speaker's face.

This approach, exemplified by Txt2Vid Txt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via Text, prioritizes perceptual realism over mathematical fidelity. The reconstructed video might not match the original pixels perfectly, but it looks "correct" and remains clear even when bandwidth is almost non-existent.

The Perceptual Shift

The ultimate goal of this research direction is a transition from optimizing for PSNR (Peak Signal-to-Noise Ratio)—a mathematical metric that often ignores human perception—to optimizing for how we actually see.

As Radu Timofte and Ren Yang demonstrated through the NTIRE challenges NTIRE 2021 Challenge on Quality Enhancement of Compressed Video, the most successful models are those that use frequency-domain information and adversarial losses to restore the "crispness" of a video, even if the resulting pixels differ slightly from the source. By embracing the generative power of neural networks, the field is moving toward a future where "low bandwidth" no longer means "low quality."

Papers in this version

Showing 60 of 75 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_neural_artifact_reduction_for_compressed_9d42c86d,
  title = {Neural Artifact Reduction for Compressed Video},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-artifact-reduction-for-compressed-video/rdr_9d42c86d89b94f5abf7f002a02622917},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_neural_artifact_reduction_for_compressed_9d42c86d_v1,
  title = {Neural Artifact Reduction for Compressed Video},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-artifact-reduction-for-compressed-video/rdr_9d42c86d89b94f5abf7f002a02622917/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}