MovieRoll: A Systematic Framework for Rolling Out Long-Form Videos

Yunyang Ge, Weijie Kong, Ryan Xu, Yue Wu, Qi Tian, Jiangfeng Xiong, Bin Lin, Mingzhe Zheng, Shenghai Yuan, Zheyuan Liu, Jianwei Zhang, Li YuanarXiv 2026 · arXiv

TL;DR

MovieRoll is a comprehensive framework designed to generate high-quality, long-form videos (up to 60 seconds) by bridging the gap between short-clip training and long-horizon inference. By introducing novel streaming fine-tuning, hierarchical cache compression, and a "cumulative" distillation strategy, it achieves state-of-the-art results on long-video benchmarks while slashing memory costs by up to 78%.

What Problem Does This Paper Address?

Generating long videos is a massive challenge for current AI models. Most high-quality video models are "bidirectional," meaning they process the entire video at once. This leads to two major bottlenecks:

  1. Computational Cost: Memory and compute requirements explode as video length increases.
  2. Latency: You have to wait for the whole video to finish denoising before seeing the first frame.

"Streaming" or autoregressive generation—producing the video chunk by chunk—is a natural solution, but it introduces exposure bias. Small errors in early chunks accumulate, leading to "drift" where the subject's identity or the scene's logic falls apart over time. Furthermore, the KV-cache (the memory used to store historical context) grows linearly, eventually exceeding GPU capacity. MovieRoll tackles these issues head-on to enable stable, minute-long video synthesis.

MovieRoll generates a minute-long video with continuous subject identity across six events.

Methods

The authors propose a "base-to-distilled" framework that evolves a standard video model into a robust long-form generator through several key innovations:

  • Window-wise Teacher Forcing (WinTF): Instead of training on isolated chunks, WinTF trains the model on rolling windows. Within a window, chunks interact bidirectionally, but they only have "causal" (one-way) access to the preceding clean history. This teaches the model to transition smoothly between events.
  • Hierarchical KV-cache Compression (HieraKV): The team discovered that different layers in a transformer "care" about history differently. Shallow layers need the full history to maintain global structure, while deeper layers primarily focus on the "sink" (the very first frames) and the most recent frames. HieraKV exploits this, retaining full history only where it matters, which reduces memory usage by 78%.
  • Cumulative Long Tuning with DMD (CuLT-DMD): To make the model fast, they distill the multi-step base model into a 5-step generator. Unlike previous methods that update parameters after every chunk, CuLT-DMD holds parameters fixed during a long "self-rollout" and averages gradients across the entire sequence. This significantly improves long-term stability.
  • AnnealCFG: Classifier-free guidance (CFG) is essential for quality but can cause the model to drift "off-manifold" during long sequences. MovieRoll gradually reduces (anneals) the guidance strength as it approaches the final clean image of each chunk.
  • Infrastructure: They implemented Flash Block-wise Causal Attention (FBCA) for efficient training and a Parallel Streaming VAE (PS-VAE) to decode video chunks across multiple GPUs with minimal latency.
Overview of the MovieRoll framework, showing WinTF, HieraKV, and CuLT-DMD.

Datasets

  • Training: An internal large-scale video corpus consisting of 620,000 single-event clips and 560,000 multi-event clips.
  • Evaluation: The primary benchmark is HeliosBench, which evaluates long-video quality, dynamics, and temporal drift. They also use prompts from MovieGen, VBench, and MemFlow for broader testing.

Key Insights and Ideas

  • Layer-Dependent History: The most striking "aha moment" is the visualization of attention patterns (Figure 4). The fact that deep layers naturally "compress" history allows for aggressive memory savings without sacrificing quality.
  • Gradient Averaging Matters: In long-rollout distillation, updating the model too frequently (per-window) can be unstable. Accumulating and averaging gradients over a full rollout (CuLT-DMD) acts as a regularizer that keeps the "story" on track.
  • Guidance as a Double-Edged Sword: While high CFG makes individual frames look better, it acts as a source of cumulative error in streaming. Annealing it is a simple but effective fix for long-horizon stability.
Layer-dependent attention patterns showing that deeper blocks concentrate more on sink and recent chunks.

Key Results

  • State-of-the-Art Performance: MovieRoll-Distilled achieved the highest total scores on HeliosBench across 10s, 30s, and 60s evaluations, outperforming competitors like LongLive and Helios-Base.
  • Efficiency Gains: HieraKV reduced peak denoising-step latency by ~53% and KV-cache memory by 78% compared to full-history caching.
  • Decoding Speed: The PS-VAE achieved a 3.97x speedup using 8 GPUs compared to a single-GPU setup, enabling much faster streaming output.
  • Human Preference: In a user study with 59 professionals, MovieRoll-Distilled was preferred over all baselines, with win rates ranging from 67% to over 90%.

Limitations and Future Work

Despite the impressive results, MovieRoll has not yet achieved real-time generation. On 8 NVIDIA H20 GPUs, it generates a 60s video in about 152 seconds (~9.5 FPS), falling short of the 24 FPS required for live playback.

The authors also note that while the model handles subject consistency well, it still struggles with extremely complex motion and camera trajectories. They suggest that future work could involve "agentic" generation, where the model uses visual feedback to plan and adjust its "story" as it unfolds, rather than just following a fixed prompt.

Summary updated

Abstract

Streaming video generation must sustain quality beyond the training horizon while handling self-generated history and growing cache costs. To address these challenges, we present MovieRoll, a base-to-distilled framework that connects streaming fine-tuning, long-rollout distillation, and long-horizon inference. We first establish a robust multi-step base through Window-wise Teacher Forcing (WinTF), which jointly denoises chunks at progressive noise levels with bidirectional attention within each window and causal access to preceding history. To retain historical context efficiently, hierarchical KV-cache compression (HieraKV) exploits layer-wise attention patterns, retaining full history in shallow layers and sink--recent history in deeper layers. This reduces cache memory by up to 78% while improving generation quality. Building on this base, cumulative long tuning with distribution matching distillation (CuLT-DMD) holds parameters fixed throughout each long self-rollout and averages local gradients before updating. Each update therefore combines supervision across temporal positions while retaining only one local computation graph, enabling efficient long-rollout training of a few-step generator. To extend generation beyond the training horizon, KV recomputation bounds context for both models, while guidance annealing (AnnealCFG) reduces guidance toward the clean endpoint to stabilize MovieRoll-Base. Complementing these modeling advances, FBCA attention and PS-VAE decoding support efficient training and streaming output. On HeliosBench, MovieRoll-Base remains competitive at 30s after training on videos of up to 15s, and MovieRoll-Distilled achieves the highest total scores among the compared methods across single- and multi-event evaluations up to 60s, combining sustained naturalness with active motion.
BibTeX
@misc{ge2026movieroll,
  author = {Yunyang Ge and Weijie Kong and Ryan Xu and Yue Wu and Qi Tian and Jiangfeng Xiong and Bin Lin and Mingzhe Zheng and Shenghai Yuan and Zheyuan Liu and Jianwei Zhang and Li Yuan},
  title = {{MovieRoll: A Systematic Framework for Rolling Out Long-Form Videos}},
  year = {2026},
  eprint = {2611.21339},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21339},
}
Figure