FastBlend: Enhancing Video Stylization Consistency via Model-Free Patch Blending

Zhongjie Duan, Chengyu Wang, Cen Chen, Weining Qian, Jun Huang, Mingyi Jin

Introduction

The advent of diffusion models like Stable Diffusion has transformed the landscape of artistic image stylization. However, extending these capabilities to video remains a significant hurdle. When frames are processed independently, minor variations in the generation process manifest as "temporal inconsistency"—a phenomenon where the video flickers or "tears" because the applied style does not move cohesively with the underlying scene geometry.

Existing approaches to mitigate this typically involve training large-scale video models, which is computationally expensive, or "zero-shot" modifications to internal attention mechanisms, which often struggle with complex motion. FastBlend introduces a robust, model-free post-processing framework that can be integrated into any image-to-video pipeline to ensure temporal smoothness.

The FastBlend Framework

FastBlend operates entirely in the image space, making it agnostic to the underlying generative model. The pipeline follows a structured three-step process:

  1. Patch Matching: Using the original "guide" video, the algorithm estimates Nearest Neighbor Fields (NNFs). Unlike standard optical flow, NNFs identify optimal patch-level matches across frames, capturing the scene's motion more effectively for stylization purposes.
  2. Image Remapping: The stylized frames are warped or remapped based on the NNFs to align with a target frame. This ensures that stylistic elements applied to an object in one frame are consistently positioned in subsequent frames.
  3. Video Blending: Aligned frames within a sliding window are blended together. This averaging process effectively cancels out the stochastic noise and flickering introduced by the diffusion model.
Figure 4: The FastBlend pipeline overview, showing the transition from inconsistent stylized frames to smooth blended frames using the original video as a motion guide.

Technical Innovations

To ensure both efficiency and visual fidelity, the authors introduced two key technical components:

  • Remapping Table: To circumvent the $O(NM)$ complexity of naive blending, a tree-like data structure was implemented. This enables a "Fast Mode" that reduces computational complexity to $O(N \log N)$.
  • Accurate Mode: A specialized alignment loss function minimizes variance between remapped pixels, preventing the blurriness or "smoggy" artifacts that can occur during simple frame averaging.

Experimental Results

The framework was evaluated using the Pixabay100 dataset, a collection of 100 diverse videos with custom stylization prompts. FastBlend demonstrated superior performance across both quantitative and qualitative metrics:

  • Temporal Consistency: FastBlend achieved a Pixel-MSE of 35.79, a significant improvement over the previous state-of-the-art, All-In-One Deflicker (53.38).
  • Efficiency: On an NVIDIA RTX 4090, FastBlend processed 100 frames in 2.27 minutes, more than twice as fast as All-In-One Deflicker (5.42 minutes).
  • User Preference: In double-blind human evaluations, FastBlend was preferred over 75% of the time against other post-processing methods and over 89% of the time against zero-shot diffusion methods.

Conclusion

FastBlend offers a highly efficient, model-agnostic solution to the persistent problem of flickering in AI-generated videos. While it faces challenges with extremely rapid motion—which can lead to texture blurring—it represents a major step forward in practical video stylization. Future work will focus on integrating more advanced motion estimation to handle high-speed action and expanding the framework's utility to general video restoration tasks.

Summary updated

Abstract

With the emergence of diffusion models and the rapid development of image processing, generating artistic images in style transfer tasks has become effortless. However, these impressive image processing approaches face consistency issues in video processing due to the independent processing of each frame. In this paper, we propose a powerful, model-free approach called FastBlend to address the consistency problem in video stylization. FastBlend functions as a post-processor and can be seamlessly integrated with diffusion models to create a robust video stylization pipeline. Based on a patch-matching algorithm, we remap and blend the aligned content across multiple frames, thus compensating for inconsistent content with neighboring frames. Moreover, we propose a tree-like data structure and a specialized loss function, aiming to optimize computational efficiency and visual quality for different application scenarios. Extensive experiments have demonstrated the effectiveness of FastBlend. Compared with both independent video deflickering algorithms and diffusion-based video processing methods, FastBlend is capable of synthesizing more coherent and realistic videos.
BibTeX
@inproceedings{duan2025fastblend,
  author = {Zhongjie Duan and Chengyu Wang and Cen Chen and Weining Qian and Jun Huang and Mingyi Jin},
  title = {{FastBlend: Enhancing Video Stylization Consistency via Model-Free Patch Blending}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1119},
}
Figure