Unity in Diversity: Video Editing via Gradient-Latent Purification

Junyu Gao, Kunlin Yang, Xuan Yao, Yufan HuCVPR 2025 · DOI

Abstract

Text-driven video editing has seen significant progress with the advent of diffusion models. However, iterative optimization-based methods often struggle with gradient noise and the lack of a clear termination criterion, leading to blurry results or over-optimization artifacts. We present Gradient-Latent Purification (GLP), a framework designed to stabilize the optimization process. By leveraging Singular Value Decomposition (SVD) to extract consistent update directions from noisy gradients and implementing a final latent purification stage, GLP produces high-fidelity, temporally consistent videos. Experimental results demonstrate that GLP outperforms existing state-of-the-art methods in both text alignment and visual quality.

Introduction

Recent advancements in text-to-video generation have paved the way for sophisticated video editing techniques. A prominent category of these techniques involves backward-based methods, which iteratively optimize a video's latent representation to minimize a loss function, such as the Delta Denoising Score (DDS). While these methods offer flexibility, they are plagued by two primary issues: gradient noise and uncontrollability.

Gradient noise arises because the self-supervised objectives used in optimization are inherently imperfect. As noise accumulates over iterations, the edited content often becomes "vague," blurry, or oversaturated. Uncontrollability refers to the difficulty in selecting the optimal number of iterations. Stopping too early results in incomplete edits, while stopping too late introduces artifacts and distorts the original video's structure.

Methodology: The Gradient-Latent Purification Framework

To address these challenges, we propose the Gradient-Latent Purification (GLP) framework. GLP introduces two main purification stages into the optimization loop: Gradient Purification (GP) and Latent Purification (LP).

The GLP framework architecture, illustrating the Gradient Purification (GP) and Latent Purification (LP) modules integrated into the optimization loop.

Gradient Purification (GP)

The GP module aims to filter out random noise from the optimization steps. Instead of applying a single noisy gradient update, we collect gradients over a short window of iterations. We then apply Singular Value Decomposition (SVD) to these collected gradients to construct a local coordinate system. By identifying the axes with the highest consistency (low variance), we determine a "concordant" update direction. This process effectively isolates the underlying editing signal from stochastic noise.

To ensure the purified updates remain within the manifold expected by the diffusion model, we apply a Gaussian Constraint. This regularization term constrains the skewness and kurtosis of the update direction, preventing the latents from drifting into regions of the latent space that the model cannot decode effectively. Furthermore, Temporal Smoothing is achieved by linking the coordinate systems of adjacent frames, ensuring that the purification process is spatially and temporally coherent.

Latent Purification (LP)

Even with purified gradients, the "optimal" stopping point remains elusive. The LP module addresses this by analyzing the last few versions of the optimized latent. Rather than selecting the final iteration, which might be over-optimized, we perform an SVD-based aggregation of these final candidates to find a stable, "purified" final state. This ensures a robust result that preserves the structural integrity of the source video while fulfilling the target prompt.

Experimental Results

We evaluated the GLP framework on the DAVIS and WebVid datasets, covering a wide range of editing tasks including object transformation, style transfer, and background modification.

Quantitative Analysis

GLP demonstrates superior performance across several key metrics. It achieved a Text-Align score of 28.67 and a Frame-Acc of 0.84, indicating high fidelity to the text prompts. In terms of temporal consistency, GLP reached a Motion-TC score of -5.45, outperforming baselines like TokenFlow and RAVE.

Qualitative Comparison

Qualitative results highlight GLP's ability to maintain sharp details and temporal stability where other methods fail.

Qualitative comparison of GLP against state-of-the-art baselines. GLP consistently produces sharper results with better adherence to the target prompt while maintaining the source video's structure.

As shown in the comparisons, methods like TokenFlow or RAVE often introduce "ghosting" artifacts or fail to fully transform the subject. In contrast, GLP successfully modifies complex subjects—such as transforming a squirrel into a robot squirrel or a bear into a red tiger—while keeping the background and fine details (like water ripples or rock textures) intact.

Conclusion and Future Work

The Gradient-Latent Purification framework provides a robust solution to the noise and uncontrollability issues inherent in optimization-based video editing. By treating consistency as a signal to be extracted via SVD, GLP achieves a new state-of-the-art in video editing quality.

Future work will focus on handling more complex 3D orientations and extreme motion blur. Integrating explicit 3D priors or pose-aware modules could further enhance the model's ability to perform structural edits in dynamic scenes.

Summary updated

BibTeX
@inproceedings{gao2025unity,
  author = {Junyu Gao and Kunlin Yang and Xuan Yao and Yufan Hu},
  title = {{Unity in Diversity: Video Editing via Gradient-Latent Purification}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02179},
}
Figure