TL;DR
BATMoS (Budget-Aware Temporal Mixture of Strategies) is a simple, training-free method to speed up video diffusion models by intelligently allocating the computational budget for Classifier-Free Guidance (CFG). By selectively reusing guidance in timesteps where its impact is minimal, BATMoS reduces denoiser evaluations by 20% while maintaining nearly identical video quality to full-cost models.
What Problem Does This Paper Address?
Generating high-quality video with diffusion models is computationally expensive. A major culprit is Classifier-Free Guidance (CFG), a technique that significantly improves prompt adherence but doubles the workload by requiring two model evaluations (one conditional, one unconditional) at every single denoising step.
Researchers usually try to save time by reducing the total number of sampling steps, but this often leads to a noticeable drop in video smoothness or detail. The authors of BATMoS ask a different question: If we have a fixed computational budget, where along the denoising timeline should we spend it? They argue that not all guidance evaluations are equally valuable, and blindly applying the same "cheap" approximation at every step is inefficient.
Methods
The core of BATMoS is a magnitude-based temporal schedule. The authors observed that the "guidance magnitude"—the difference between the conditional and unconditional predictions—varies significantly over the course of generating a video.
Instead of cutting sampling steps, BATMoS keeps the full sampling grid (e.g., 50 updates) but chooses between two strategies for each step:
- Full CFG: Compute both branches fresh (Cost: 2 evaluations).
- Guidance Reuse: Compute only the conditional branch and combine it with a "cached" guidance difference from a previous step (Cost: 1 evaluation).
To decide which steps get the "budget" of a full CFG update, BATMoS partitions the trajectory into blocks and ranks them based on their average guidance magnitude. Blocks with the lowest magnitude are assigned to reuse guidance first. This ranking is pre-computed once and works across different prompts and seeds.
Furthermore, BATMoS can be composed with feature reuse (like TeaCache), which skips internal transformer layers within a single evaluation branch. This allows for even deeper computational savings.
Datasets
The researchers evaluated BATMoS using the CogVideoX-2B model. To measure performance, they used VBench, a comprehensive benchmark for video generation. Their evaluation involved generating 4,720 videos across 944 unique prompts, covering all sixteen official VBench dimensions (such as imaging quality, temporal flickering, and motion smoothness).
Key Insights and Ideas
- Guidance Importance is U-Shaped: The "aha moment" is the realization that guidance magnitude is high at the very beginning (establishing structure) and rises again at the very end (refining details), but dips significantly in the middle.
- Preserving the Sampling Grid: By reducing the cost per step rather than the number of steps, BATMoS maintains the temporal dynamics of the original model much better than simple step-skipping.
- Single Integer Control: The entire computational budget is controlled by a single integer ($k$), which determines how many low-importance blocks are relegated to guidance reuse. This eliminates the need for complex hyperparameter tuning or per-budget threshold searches.
- Complementary Mechanisms: Guidance reuse (approximating the difference between branches) and feature reuse (approximating internal network layers) target different redundancies, making them highly effective when used together.
Key Results
BATMoS achieves state-of-the-art efficiency for video diffusion:
- Near-Lossless 20% Speedup: Using 80 branch evaluations instead of 100, BATMoS achieved a VBench total score of 76.97, coming within a tiny 0.10 points of the full CFG score (77.07).
- Superior to Baselines: It outperformed other common strategies like "front-loading" guidance or simply reducing the number of sampling steps to 40.
- Effective Composition: When combined with feature reuse, it reduced the effective computation to just 62 branch evaluations with only a 0.11-point drop in total score compared to the 80-branch version.
- Qualitative Consistency: As shown in Figure 1, the visual difference between full CFG and BATMoS is virtually indistinguishable to the human eye across various motion types and styles.
Limitations and Future Work
While highly effective, the authors note a few areas for further exploration:
- Fixed Ranking: The current magnitude ranking is fixed based on a set of reference prompts. While it generalizes well, a dynamic, prompt-specific router might yield even better results.
- Budget Resolution: The budget is currently controlled in blocks of 5 updates. Finer-grained control could allow for more precise optimization.
- Model Specificity: The study focused on CogVideoX-2B; testing the magnitude-based schedule on a wider variety of architectures (like Mamba-based models) would confirm its universality.
Summary updated