When to Start Denoising? Adaptive Denoising Entry for Generative Video Compression

Zifu Zhang, Yuqiao Li, Ding Ding, Shengxi Li, Mai Xu, Daowen Li, Kai Li, Ying ChenarXiv 2026 · arXiv

TL;DR

ADE-GVC is the first "timestep-aware" generative video compression framework that adaptively selects when to start the diffusion denoising process based on a video's specific distortion level. By moving away from "one-size-fits-all" fixed entry points, it achieves a superior balance between reconstruction fidelity and perceptual realism, delivering state-of-the-art performance at extremely low bitrates.

What Problem Does This Paper Address?

Generative video compression, particularly using diffusion models, has become a leading paradigm for extreme low-bitrate scenarios. These models can "hallucinate" realistic textures where traditional codecs would simply produce a blurry mess. However, their effectiveness hinges on a delicate trade-off: fidelity vs. realism.

Current diffusion-based codecs typically start the denoising process from a compressed representation perturbed by a fixed amount of noise. This is problematic. If you inject too much noise, the model has too much "creative freedom" and loses fidelity to the original source. If you inject too little, the model can't effectively "clean up" the compression artifacts, leaving the video looking unrealistic. Because every video sequence degrades differently under compression, a fixed entry point is inherently sub-optimal.

Figure 1. Illustration of the adaptive denoising entry scheme on MCL-JCV video sequences [33], in which fixed start timestep fails to restore various compression degradations. In contrast, we pro- pose to optimize for adaptive entry timestep that is capable of ad- dressing both fidelity and realism for generative video compres- sion.

Methods

The authors propose ADE-GVC (Adaptive Diffusion Entry for Generative Video Compression), which introduces three key components:

  1. Geometric Optimization Formulation: The researchers mathematically formulated the denoising entry selection as an optimization problem. Their analysis revealed a fundamental principle: the optimal entry timestep should decrease monotonically as compression distortion increases. In other words, more heavily distorted videos need an earlier entry into the denoising process (more noise injection) to be properly restored.
  2. Timestep Predictor (TP): A lightweight module that analyzes the compressed latent representation to estimate the "Goldilocks" denoising start point ($\tau$) for that specific sequence.
  3. Temporal Adaptor (TA): To ensure temporal consistency—a common headache in generative video—this module extracts compression-aware spatio-temporal features and injects them as conditions into the diffusion model.

The framework is built upon a pre-trained video diffusion backbone (Wan2.1-VACE) and uses a three-stage training strategy to align the compression network, the diffusion model, and the new adaptive predictor.

Figure 2. Geometric analysis showing how an adaptive strategy (b) can accommodate varying distortion levels better than a fixed strategy (a).

Datasets

  • Training: A subset of the Pexels dataset, comprising approximately 480,000 high-quality video clips selected for aesthetic and clarity scores.
  • Evaluation: Standard benchmarks including HEVC Class B, UVG, and MCL-JCV. These datasets provide a diverse range of motion and texture complexities to test the robustness of the codec.

Key Insights and Ideas

  • The "Aha!" Correlation: The authors found a remarkably strong negative correlation (coefficient of -0.84) between their predicted start timesteps and the DISTS perceptual metric. This empirically validates their theoretical model: videos that the metric identifies as "more distorted" are exactly the ones the predictor chooses to denoise more aggressively.
  • Distortion, Not Content: Interestingly, the predicted entry point is primarily driven by compression-induced distortion rather than the intrinsic characteristics of the source video (like brightness or color richness). This suggests the model is truly learning to "see" and react to the artifacts themselves.
  • Efficiency through Adaptation: By shortening the generative refinement trajectory for low-distortion videos, the adaptive approach doesn't just improve quality—it avoids unnecessary computation.

Key Results

ADE-GVC sets a new bar for generative video compression:

  • Massive Bitrate Savings: Compared to the latest open-source baseline (GLC-Video), ADE-GVC achieves average BD-rate savings of 40.99% on the DISTS perceptual metric and a staggering 95.24% on PSNR.
  • Perceptual Superiority: It significantly outperforms existing methods like S²VC and GNVC-VD across all tested datasets in terms of LPIPS and DISTS.
  • Visual Quality: Qualitatively, the method excels at recovering fine textures (like wall grains) and maintaining sharp, recognizable text boundaries that other generative methods often distort or hallucinate into nonsense.
  • Speed: Through optimization strategies like parallel inference and BF16 precision, the authors achieved a 5.11x speedup, bringing decoding time down to ~0.83s per 1080p frame on an NVIDIA H20 GPU.

Limitations and Future Work

While ADE-GVC adaptively chooses where to start denoising, it currently uses a fixed number of sampling steps for every sequence. The authors acknowledge that a truly optimal system would likely adapt both the entry point and the number of steps to further boost efficiency.

Additionally, while this work focuses on video, the underlying theory of adaptive denoising entry is broadly applicable and could likely benefit generative image compression and other low-level vision tasks like super-resolution or dehazing.

Summary updated

Abstract

Generative video compression offers a promising solution for compressing and reconstructing videos under extremely low bitrate constraints, among which the diffusion-based solutions still achieve the best performance by providing the powerful generative latent prior. However, their effectiveness critically depends on when the compressed latent is introduced into the diffusion process, which directly governs the trade-off between reconstruction fidelity and perceptual realism. To address this limitation, we identify denoising start-timestep selection as a key factor governing this trade-off and formulate it as a geometric optimization problem. Based on this formulation, we propose a novel adaptive diffusion entry method for generative video compression (ADE-GVC), the first timestep-aware generative video compression framework, featuring a lightweight predictor that estimates an adaptive denoising start point from the compressed latent. A temporal adapter further extracts compression-aware conditions and injects them into a pretrained video diffusion model, enabling the reconstruction of videos with both high fidelity and realism. More importantly, we observe a strong quantitative relationship between the predicted timesteps and the distortion between the original and compressed latents, providing empirical support for our optimization formulation. Extensive experiments demonstrate that ADE-GVC achieves the state-of-the-art performance on perceptual metrics and substantially improves reconstruction realism over existing generative video compression methods. Notably, our ADE-GVC achieves an average bitrate savings of 40.99\% (on DISTS) and 95.24\% (on PSNR) compared to the latest open-source baseline GLC-Video.
BibTeX
@misc{zhang2026when,
  author = {Zifu Zhang and Yuqiao Li and Ding Ding and Shengxi Li and Mai Xu and Daowen Li and Kai Li and Ying Chen},
  title = {{When to Start Denoising? Adaptive Denoising Entry for Generative Video Compression}},
  year = {2026},
  eprint = {2611.13620},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13620},
}
Figure