Abstract
Significant progress has been achieved in high-fidelity video synthesis, yet current paradigms often fall short in effectively integrating identity information from multiple subjects. This leads to semantic conflicts and suboptimal performance in preserving identities and interactions, limiting controllability and applicability. To tackle this issue, we introduce ID-Crafter, a framework for multi-subject video generation that achieves superior identity preservation and semantic coherence. ID-Crafter integrates three key components: (i) a hierarchical identity-preserving attention mechanism that progressively aggregates features at intra-subject, inter-subject, and cross-modal levels; (ii) a semantic understanding module powered by a pretrained Vision-Language Model (VLM) to provide fine-grained guidance and capture complex inter-subject relationships; and (iii) an online reinforcement learning phase to further refine the model for critical concepts. Furthermore, we construct a new dataset to facilitate robust training and evaluation. Extensive experiments demonstrate that ID-Crafter establishes new state-of-the-art performance on multi-subject video generation benchmarks, excelling in identity preservation, temporal consistency, and overall video quality.
Introduction
While text-to-video models have reached impressive levels of realism, generating videos with multiple specific subjects remains a significant challenge. Most existing models suffer from "identity leakage"—where features of one subject bleed into another—and "semantic conflicts," where the model fails to correctly assign actions to the intended subjects. ID-Crafter addresses these issues by moving beyond simple feature concatenation, instead employing a structured approach to identity preservation and semantic grounding.
Methodology
ID-Crafter is built upon a latent diffusion transformer (DiT) backbone and introduces three primary innovations:
1. Hierarchical Identity-Preserving Attention
To prevent identity bleed, ID-Crafter processes subject tokens through a three-stage attention hierarchy:
- Intra-subject attention: Refines individual subject features in isolation to capture fine-grained details.
- Gated Inter-subject attention: Models the relationships and interactions between subjects while strictly maintaining identity boundaries.
- Cross-modal attention: Fuses these refined subject features with text and video tokens to ensure the final output aligns with the global prompt.
2. VLM-Based Semantic Grounding
Traditional text encoders like CLIP often lack the depth required for complex, multi-subject prompts. ID-Crafter utilizes a pretrained Vision-Language Model (Qwen2.5-VL) as a dual text-image encoder. This allows the model to perform granular, cross-modal reasoning, effectively acting as a semantic router that guides the spatial and temporal layout of the generated video.
3. Online Reinforcement Learning (Flow-GRPO)
To optimize the balance between identity fidelity and aesthetic quality, the framework employs an online RL post-training stage. Using a novel algorithm called Flow-GRPO, the model is fine-tuned based on a composite reward function that evaluates face similarity, subject consistency, and overall physical plausibility. By utilizing groupwise comparisons rather than a value network, Flow-GRPO provides a stable and effective way to refine video diffusion models.
Data and Training
The authors curated a massive dataset to support complex multi-subject training, including the OpenS2V-Nexus collection (218k videos) and a high-quality internal collection of 9.3k professionally shot videos. To avoid the artifacts of simple data augmentation, they used an advanced VLM pipeline to synthesize coherent multi-subject reference images, providing the model with rich priors for appearance and interaction.
Key Results and Evaluation
ID-Crafter sets a new state-of-the-art for multi-subject video generation:
- Benchmark Performance: The 14B parameter version achieved a total score of 57.05% on the OpenS2V-Nexus benchmark, outperforming both open-source (Phantom, SkyReels) and proprietary (Kling 1.6, Vidu) models.
- Impact of RL: The online RL stage significantly boosted performance, yielding a 13.7% increase in face similarity and a 14.9% increase in aesthetic scores over the supervised fine-tuning baseline.
- Human Preference: In user studies, ID-Crafter was preferred 60% of the time for identity consistency and 65% of the time for motion naturalness compared to leading competitors.
Conclusion
By combining hierarchical attention, VLM-driven semantic grounding, and stable online reinforcement learning, ID-Crafter effectively solves the core problems of identity leakage and semantic confusion in multi-subject video generation. While challenges remain in modeling extremely complex physical interactions, ID-Crafter represents a major step forward in controllable and compositional video synthesis.
Summary updated