TL;DR
Wave Forcing (WF) is a co-designed framework that achieves "speed-of-light" streaming video generation by pipelining multi-step denoising across multiple GPUs. By allowing newer video chunks to "look ahead" at cleaner, partially denoised versions of preceding chunks, it achieves a throughput of one completed chunk per forward pass (1F1C), delivering up to 134.8 end-to-end FPS and superior long-video stability compared to existing autoregressive methods.
What Problem Does This Paper Address?
Generating long, high-quality videos in real-time is a massive computational challenge. Most current state-of-the-art models use chunk-wise autoregressive generation, where the video is generated in small segments (chunks).
However, these methods face two major hurdles:
- Efficiency Bottlenecks: Standard methods require at least two forward passes per chunk—one for denoising and one to "store" the clean result into a Key/Value (K/V) cache for future chunks. This results in an "NF1C" (N Forward rounds for 1 Chunk) schedule, which is inherently slow.
- Quality Degradation: As the video gets longer, errors accumulate. This often leads to "camera drift" (the background moves unnaturally) or "subject loss" (the main character disappears or transforms into something else).
Methods
The authors propose a holistic co-design of the generation schedule, the training process, and the hardware execution.
1. Causal Wavefront Scheduling
Instead of waiting for chunk $A$ to be fully denoised before starting chunk $B$, Wave Forcing uses a staggered pipeline. In any given "round," different GPUs work on different stages of denoising for different chunks.
- GPU 1 might be doing the first denoising step for Chunk 5.
- GPU 2 is doing the second step for Chunk 4.
- GPU 3 is doing the final "store" pass for Chunk 1.
This creates a "wavefront" that moves through the video. Crucially, they use chunk-wise causal attention, allowing a noisy chunk to attend to earlier chunks that are "cleaner" (further along in their denoising process), even if they aren't finished yet.
2. Three-Stage Training
To make the model robust to this "mixed-noise" context, the authors developed a specialized training recipe:
- Stage 1: Mixed-noise initialization. Teaches the model to handle bidirectional attention across chunks at different noise levels.
- Stage 2: Off-policy Distillation (DMD). Uses teacher videos to provide strong motion and subject guidance.
- Stage 3: Causal Adaptation. Fine-tunes the model on its own self-generated history to prevent error accumulation during long rollouts.
3. Wave Runtime (WaveRT)
To translate this theoretical scheduling into actual speed, the authors built WaveRT. It uses Wave Parallelism (WP) to overlap communication and computation. As soon as a "producer" GPU finishes calculating K/V values for a layer, it immediately sends them to the "consumer" GPUs in the pipeline, rather than waiting for the entire forward pass to finish.
Datasets
The researchers evaluated their methods primarily using:
- Wan2.1-T2V-1.3B: The base Diffusion Transformer (DiT) model used for implementation.
- VBench & VBench-Long: Standard benchmarks for video quality, semantic consistency, and motion.
- MovieGen Prompts: A set of 200 prompts used to test long-video (100-second) generation.
- Subject-focused Subset: A newly curated 100-prompt subset specifically designed to test "Subject Preservation" and "Camera Drift" over long durations.
Key Insights and Ideas
- Intermediate States are Useful Context: The core "aha" moment is that a partially denoised chunk is a "good enough" conditioning signal for the next chunk's earlier denoising stages. You don't need to wait for perfection to maintain temporal flow.
- 1F1C is the Efficiency Limit: By assigning a full-model replica to each denoising stage, the system reaches the theoretical limit of one chunk output per concurrent forward round.
- Co-design is Mandatory: You cannot simply take a standard model and run it in a wavefront pipeline; it will produce garbage. The training must be aware of the specific causal mask and noise distribution used during inference.
Key Results
- Throughput: On eight NVIDIA H200 GPUs, 4-step Wave Forcing achieved 134.8 end-to-end FPS, a 7.07x speedup over a single-GPU baseline.
- System Efficiency: Wave Parallelism outperformed traditional context-parallel baselines (like Ulysses or Ring Attention) by up to 2.53x across various model scales (1.3B to 14B parameters).
- Long-Video Stability: On 100-second videos, WF maintained a Subject Preservation score of 97.10 (compared to 92.75 for Rolling Forcing) and significantly higher "Dynamic Degree," meaning the videos had more natural, active motion without losing the subject.
Limitations and Future Work
- Training Complexity: The current multi-stage training process is complex and computationally expensive. Simplifying this is a priority.
- Fixed Sampling Budget: The degree of parallelism is currently tied to the number of denoising steps. If you want a 50-step high-quality render, you'd need 50+ GPUs to maintain the 1F1C throughput.
- Interactive Latency: While throughput (FPS) is high, the initial latency (time to first frame) is still limited by the pipeline fill time.
- Future Directions: The authors aim to extend this to interactive world models, where user inputs (like camera controls) can affect the "in-flight" chunks of the wavefront in real-time.
Summary updated