Abstract
Restoring low-quality video into high-fidelity, photorealistic content remains a significant challenge in computer vision. While generative Text-to-Video (T2V) models have demonstrated remarkable synthesis capabilities, adapting them for restoration often leads to "distribution drift," where models lose their generative priors when fine-tuned on degraded data. We introduce Vivid-VR, a framework that utilizes a novel concept distillation strategy and a redesigned ControlNet architecture to bridge this gap. By aligning training samples with the model's internal generative concepts, Vivid-VR restores videos with superior texture realism and temporal stability, outperforming current state-of-the-art methods on both real-world and AI-generated benchmarks.
Introduction
Video restoration is an inherently ill-posed problem characterized by the need to reconstruct missing high-frequency details from degraded inputs. Traditional methods often produce over-smoothed or "plastic" textures to avoid artifacts. Recent attempts to leverage powerful generative models for this task have encountered the "alignment problem": the gap between the model's high-quality training distribution and the messy reality of low-quality video. This often results in flickering artifacts or a loss of the model's original generative "imagination."
Methodology
Vivid-VR addresses these challenges through three key architectural and procedural innovations designed to preserve the "vividness" of the original generative model while maintaining strict fidelity to the input video.
1. Concept Distillation Training
To combat distribution drift, Vivid-VR employs a "concept distillation" strategy. Instead of training solely on standard low-quality/high-quality pairs, the authors use a pre-trained T2V model (CogVideoX) to generate "distilled" versions of training videos. By adding noise to a source video and then denoising it using the T2V model guided by text prompts, they create targets that are perfectly aligned with the model's internal conceptual space. Blending these distilled samples into the training process ensures the model retains its ability to generate realistic textures during restoration.
2. Architectural Enhancements
The framework introduces a Control Feature Projector, a lightweight CNN-based module that filters degradation artifacts from input video latents before they enter the main pipeline. This prevents the model from inadvertently "restoring" noise. Furthermore, a Dual-branch ControlNet Connector is utilized. This module uses an MLP branch for basic feature mapping and a Cross-Attention branch to dynamically retrieve structural details from the control signal, ensuring the output remains faithful to the original video's geometry.
Experiments and Results
The model was trained on a curated dataset of 500,000 high-quality text-video pairs derived from a massive pool of 3 million HD videos. Evaluation was conducted across synthetic (SPMCS, YouHQ40), real-world (VideoLQ, UGC50), and AI-generated (AIGC50) benchmarks.
Quantitative Performance
Vivid-VR consistently achieves top rankings in no-reference quality metrics such as NIQE and MUSIQ. On the UGC50 real-world dataset, it achieved a NIQE score of 4.361, a significant improvement over competing methods like MGLD (4.633). These scores reflect the model's ability to produce natural-looking images that lack the artificial "hallucinations" common in other generative restorers.
Qualitative Comparison
Visually, Vivid-VR excels in rendering fine-grained textures such as human hair, architectural details, and complex landscapes. Unlike image-based restorers that often suffer from inter-frame flickering, Vivid-VR maintains high temporal consistency, producing stable and fluid video outputs.
Conclusion
Vivid-VR represents a significant step forward in generative video restoration by successfully decoupling the "cleaning" of noise from the "creation" of detail. By distilling concepts from large-scale Diffusion Transformers, it achieves a level of photorealism previously unattainable in the field. While the current 5-billion parameter architecture requires substantial computational resources, future work will focus on efficiency through "one-step" diffusion techniques to bring these high-quality restoration capabilities to consumer-grade hardware.
Summary updated