VidEvo: Evolving Video Editing through Exhaustive Temporal Modeling

Sizhe Dang, Huan Liu, Mengmeng Wang, Xin Lai, Guang Dai, Jingdong WangIJCAI 2025 · DOI

Abstract

Text-guided video editing (TGVE) aims to modify source videos based on textual prompts while preserving original motion and structure. However, existing methods often struggle with temporal flickering and loss of fine details. We present VidEvo, a novel one-shot video editing framework that introduces Null-Video Embedding (NVE) to capture global temporal structure and Window-Frame Attention (WFA) to ensure local continuity. By providing explicit temporal cues, VidEvo achieves superior temporal consistency and visual quality compared to state-of-the-art baselines on the DAVIS benchmark, maintaining competitive inference efficiency.

Introduction

The primary challenge in text-guided video editing is maintaining consistency across frames. When adapting pre-trained Text-to-Image (T2I) diffusion models to video, two main issues arise: temporal inconsistency, where objects "jump" or flicker, and poor reconstruction, where fine details are lost or backgrounds drift. Current methods often rely on implicit tuning of attention modules, which lacks explicit temporal guidance. VidEvo addresses this by moving toward "exhaustive" temporal modeling, deriving explicit cues directly from the source video.

Methodology

VidEvo operates in two distinct phases: Reconstruction and Editing. It introduces two core technical innovations:

Null-Video Embedding (NVE)

While traditional null-text inversion optimizes prompts for image reconstruction, NVE learns multi-scale temporal embeddings in the visual space. These embeddings capture the global "skeleton" and timing of the source video. During editing, these embeddings are injected into the U-Net’s attention modules, serving as a global anchor to ensure structural consistency.

Window-Frame Attention (WFA)

To manage computational complexity and reduce noise, WFA restricts the self-attention mechanism to a sliding window of neighboring frames (typically 3). By focusing on immediate temporal context, WFA preserves local textures and ensures smooth transitions more effectively than full spatio-temporal attention, while also reducing GPU memory requirements.

Key Results

VidEvo was evaluated against high-profile baselines such as Tune-A-Video (TAV), Vid2vid-Zero, and Video-P2P using the DAVIS dataset.

Qualitative comparison of VidEvo against baselines. VidEvo (Ours) demonstrates superior reconstruction of the dancer's dress and background consistency compared to vid2vid and TAV.

Quantitative analysis confirms VidEvo's state-of-the-art performance:

  • Temporal Consistency: Achieved a CLIP Score of 0.969, the highest among tested methods.
  • Visual Quality: Recorded the lowest FID of 23.32, indicating more natural-looking results.
  • Efficiency: Maintains a competitive inference time of 1.5 minutes and lower memory usage (23.8 GB) than Video-P2P.
  • User Preference: Human studies showed a strong preference for VidEvo in terms of both consistency and prompt alignment.

Conclusion and Future Work

VidEvo demonstrates that explicit temporal modeling in the visual space significantly improves video editing quality. While it requires a 7.2-minute optimization phase per video, its "plug-and-play" nature allows it to enhance other existing pipelines. Future research will explore hierarchical attention mechanisms to handle long-range dependencies in extended video clips more efficiently.

Summary updated

BibTeX
@inproceedings{dang2025videvo,
  author = {Sizhe Dang and Huan Liu and Mengmeng Wang and Xin Lai and Guang Dai and Jingdong Wang},
  title = {{VidEvo: Evolving Video Editing through Exhaustive Temporal Modeling}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/99},
}
Figure