TL;DR
BlockAlign is a new preference optimization framework designed specifically for "few-step" distilled video generators. By resolving a fundamental mismatch between how these models predict motion and how they are typically fine-tuned, BlockAlign achieves significant gains in visual quality and structural plausibility while being 3x faster to train than traditional alignment methods.
What Problem Does This Paper Address?
The current frontier of video generation is moving toward trajectory distillation—techniques like Parallel Decoding Distillation (PDD) and Parametric Trajectory Distillation (PTD) that compress a 50-step denoising process into just 10 steps. While fast, these models still struggle with "human" details: extra limbs, physics-defying movements, and inconsistent identities.
To fix these, researchers usually turn to Preference Optimization (like DPO), where a model is fine-tuned on pairs of "good" vs. "bad" videos. However, there’s a catch. Standard alignment tools assume the model predicts "instantaneous velocity" at any given moment. Distilled models don't do that; they predict an average velocity over a finite "block" or "hop"—essentially a macro-jump through time.
If you try to align these models using standard methods (a naive approach the authors call DirectAlign), you only update a fraction of the model's internal heads. This creates a "training-inference mismatch" that actually makes the video quality worse than the original unaligned model.
Methods
The authors introduce BlockAlign, an interface-consistent framework that respects the "blocky" nature of distilled generators.
- Boundary-Aligned Supervision: Instead of supervising the model at arbitrary points, BlockAlign only looks at valid block boundaries. It queries the fused output of all velocity heads, ensuring the gradients flow back through the entire prediction mechanism used during actual inference.
- Multi-Regime Pipeline: The framework isn't just one algorithm; it’s a pipeline that combines:
- Offline Alignment: Using fixed pairs of videos ranked by humans.
- Semi-Online Alignment: A clever "reward-free" method that uses partial noising/denoising (SDEdit-style) to create pairs that specifically target either texture refinement or compositional layout.
- Online Alignment: Real-time rollouts where a Large Vision-Language Model (VLM) like Gemini-4-Argon acts as a judge.
- Hierarchical Preference Selection: To prevent "reward hacking" (where a model makes a pretty video that is physically impossible), they use a gatekeeper approach: a video must pass a structural/physical plausibility check before it is even ranked for "softer" qualities like color or style.
- Objective-Aware Scheduling: They recognize that different blocks in the denoising process handle different things. Early blocks control global motion and layout, while late blocks handle fine textures. BlockAlign prioritizes updates to specific blocks based on the training goal.
Datasets
The researchers evaluated BlockAlign across two massive backbones:
- RefVid-V4: An in-house 30B parameter Diffusion Transformer (DiT) trained on 5,000 human-annotated pairs and 10,000 curated reference videos.
- MiniMax-H3: A 33B parameter open-source DiT.
- Public Benchmarks: They also tested zero-shot generalization on OpenS2V-Eval (360 test cases) and OmniVBench (813 test cases), which cover everything from subject consistency to complex narrative following.
Key Insights and Ideas
- The "Aha" Moment: The realization that the "fused-head" interface of distilled models is a first-class citizen. You cannot treat a 10-step distilled model as if it were just a 10-step version of a 100-step model; the underlying math of the "macro-jump" is different.
- Efficiency as an Enabler: Because the distilled models are 5x faster to run, "online" training (which requires generating thousands of candidate videos during the training loop) finally becomes computationally practical.
- Structure First: By using a hierarchical judge, the authors found they could eliminate "nightmare fuel" artifacts (like three-legged people) much more effectively than by using a single scalar reward score.
Key Results
- Human Preference: On RefVid-V4, BlockAlign achieved a 61.8% non-tie win rate over the base distilled model. Against the naive DirectAlign method, it reached a staggering 82.7% win rate.
- Training Speed: The approach unlocked a 3x end-to-end training speedup, reducing a 1,500-step training run from an estimated 600 hours down to 200 hours on 96 H200 GPUs.
- Benchmarks: On OmniVBench, BlockAlign ranked second only to much more expensive models, outperforming Kling 3.0 Omni and Gemini Omni in several structural and content-related categories.
- Diversity: Crucially, the authors proved that this alignment doesn't lead to "mode collapse." The models still produce a diverse range of videos for the same prompt, just with higher quality.
Limitations and Future Work
- Compute Requirements: While 3x faster, it still requires a massive footprint (96 H200 GPUs), which is out of reach for most academic labs.
- Reward Model Bottleneck: The authors noted that current off-the-shelf video reward models aren't quite "precise" enough for fine-grained human-object interactions, necessitating the use of expensive VLM judges.
- Future Directions: They suggest exploring more advanced RL paradigms like GRPO (Group Relative Policy Optimization) or differentiable reward backpropagation specifically tailored for these block-based architectures.
For more details, see the full paper: "BlockAlign: Preference Optimization for Trajectory-Distilled Video Generation" (arXiv:2611.20351v1).
Summary updated