TL;DR
EFlow introduces an efficient framework to train few-step video diffusion transformers from scratch, completely bypassing the need for expensive teacher distillation. By combining a token-droppable hybrid attention mechanism with a lightweight guidance training recipe, EFlow achieves up to a 2.5× increase in training throughput and generates high-quality videos in just 4 steps, slashing inference latency by up to 45×.
What Problem Does This Problem Address?
Generating high-fidelity videos using Video Diffusion Transformers (VideoDiTs) faces a compounding "double bottleneck": heavy per-step compute due to the quadratic complexity of spatiotemporal self-attention, and the requirement of dozens of iterative sampling steps.
While few-step generation methods exist to solve the sampling bottleneck, they typically rely on complex, multi-stage teacher distillation pipelines. Training a few-step model from scratch using solution-flow matching avoids distillation but introduces massive training overhead; it requires multiple network evaluations per step to enforce consistency and compute Classifier-Free Guidance (CFG). EFlow aims to solve both bottlenecks simultaneously, making from-scratch few-step video transformer training computationally practical.
Methods
The authors introduce three core components to achieve multi-level efficiency:
- Gated Local-Global Attention (GLGA): Traditional efficient DiTs rely on convolutional layers to recover local details lost by linear attention. However, convolutions assume a dense, regular grid, making them break when tokens are randomly dropped to save compute. GLGA solves this by fusing global linear attention (for long-range context) with local sliding-window attention (for fine details) via an input-aware gating mechanism. Because it avoids grid-aligned convolutions, it is highly stable under aggressive random token-dropping (up to 75%) during training.
- Path-Drop Guided (PDG) Training: To avoid the double-forward pass typically required by CFG (conditional and unconditional branches), PDG replaces the expensive unconditional branch with a computationally cheap "weak-path" pass that simply skips the middle transformer layers.
- Mean-Velocity Additivity (MVA) Regularizer: To prevent integration errors from accumulating over the large time jumps used in few-step inference, MVA enforces a global mathematical constraint (the semigroup property) ensuring that a single large jump matches the composition of smaller intermediate steps.
Datasets
- Kinetics-700: Used for class-conditional video generation experiments to isolate architectural and objective performance.
- Proprietary Large-Scale Text-to-Video Dataset: Used to evaluate high-resolution (480p) text-to-video generation capabilities.
The models were evaluated quantitatively using Fréchet Video Distance (FVD) and the VBench benchmark suite.
Key Insights and Ideas
- Convolutions are the enemy of token dropping: Standard spatial convolutions require structured grid neighborhoods. Dropping tokens destroys this contiguity, causing catastrophic model degradation. Replacing convolutions with sliding-window attention preserves local expressivity while remaining robust to token removal.
- The Semigroup Property Fixes Few-Step Drift: In few-step generation, taking massive time jumps causes trajectory drift, leading to blur and structural collapse. Forcing the network to obey the semigroup property via the MVA regularizer acts as a virtual refinement factor, allowing a 4-step model to achieve the structural fidelity of a 50-step model.
- Unconditional paths don't need full capacity: CFG targets can be stabilized using a lightweight, block-skipped "weak" version of the model as the unconditional baseline, eliminating the need for a full second forward pass during training.
Key Results
- Inference Speedup: EFlow achieves a 45.3× lower inference latency compared to standard iterative models like Wan2.1-1.3B, generating a 480p video clip in just 3.09 seconds using only 4 inference steps.
- Training Throughput: EFlow delivers up to 2.5× higher training throughput compared to standard solution-flow baselines and 1.6× over standard flow matching.
- Quality Improvements: On Kinetics-700, upgrading to the PDG-SFM objective with the MVA regularizer improved the 4-step generation quality significantly, dropping the FVD score from 185.2 to 146.7.
Limitations and Future Work
- Scale: The framework was tested up to a 1.5B parameter model at 480p resolution. Its behavior at industry scales (10B+ parameters, 720p/1080p resolutions) remains uncharacterized.
- Lack of Pre-trained Priors: Because EFlow trains entirely from scratch, it lacks the rich "world knowledge" priors inherited by models fine-tuned from massive pre-trained text-to-image foundation models. This results in lower absolute semantic scores on open-ended benchmarks like VBench.
Future directions include adapting this sparse training framework to support the highly efficient fine-tuning of pre-trained dense models and pushing towards stable 1- or 2-step generation.
Summary updated