Abstract
Identity-preserving text-to-video (IPT2V) generation aims to synthesize high-quality videos that adhere to textual descriptions while maintaining the precise facial features of a reference image. Current approaches often struggle with the "semantic gap" between static portrait references and dynamic, context-heavy prompts, frequently requiring expensive fine-tuning or massive datasets. In this paper, we present TPIGE, a training-free framework that leverages Large Language Models (LLMs) for prompt refinement, identity-consistent image re-rendering, and a novel spatiotemporal guidance strategy. TPIGE achieved first place in the ACM Multimedia 2025 Identity-Preserving Video Generation Challenge, demonstrating superior identity fidelity and visual realism without the need for model retraining.
Introduction
The task of generating videos that maintain a specific person's identity across various actions and environments is a significant challenge in generative AI. Most existing solutions fall into two categories: per-ID fine-tuning, which is computationally expensive and slow, or offline training on massive datasets, which often fails to generalize to unseen faces.
A primary obstacle is the semantic conflict between the reference image and the target prompt. For instance, a reference photo might show a person in a formal suit, while the prompt requires them to be a lifeguard in a swimming pool. This discrepancy often leads to "identity leakage," where the model either ignores the prompt to keep the suit or loses the face's likeness to satisfy the prompt. TPIGE addresses these issues through a modular, training-free pipeline that harmonizes the inputs before and during the diffusion process.
The TPIGE Framework
The TPIGE framework consists of three core enhancement components designed to bridge the gap between identity preservation and motion quality.
1. Face-Aware Prompt Enhancement (PE)
To ensure the video generator prioritizes the correct facial features, the framework employs GPT-4o to analyze the reference face. The LLM extracts detailed descriptors—such as age, hair texture, and unique facial traits—and injects them into the original text prompt. This explicit textual guidance helps the diffusion model focus on the specific attributes that define the individual's identity.
2. Prompt-Aware Reference Image Enhancement (IE)
To resolve conflicts between the reference image's original context and the prompt's requirements, TPIGE utilizes an identity-preserving image generator (e.g., Ace++). This step "re-renders" the reference person into the attire or setting described in the prompt before video generation begins. By providing the video model with a reference image that is already semantically aligned with the scene, the complexity of the generation task is significantly reduced.
3. ID-Aware Spatiotemporal Guidance Enhancement (GE)
During the diffusion sampling process, TPIGE introduces a novel guidance mechanism. The system compares a "strong" reference-conditioned model with a "weak" reference-free model. By calculating the gradient difference between these two states, the framework can steer the denoising process. This ensures that the identity-related "noise" is preserved while allowing the motion-related components to evolve naturally, resulting in high-fidelity videos that are not static.
Implementation via Mixture-of-Experts (MoE)
Recognizing that no single enhancement strategy is optimal for every scenario, the authors implemented a Mixture-of-Experts (MoE) selection strategy. The system generates multiple candidate videos using different combinations of PE, IE, and GE. These candidates are then evaluated using automated metrics: GMEScore for text alignment and ArcScore for identity consistency. The highest-scoring video is selected as the final output, allowing the framework to adapt to the specific difficulties of each prompt-ID pair.
Experimental Results
Dataset and Evaluation
The framework was rigorously tested on the ACM Multimedia 2025 Identity-Preserving Video Generation Challenge dataset, which includes 200 unseen person IDs and 1,000 unique test pairs. TPIGE was compared against leading open-source models (VACE, Phantom) and high-performing closed-source models (Hailuo).
Performance Metrics
TPIGE secured 1st place in the challenge, outperforming competitors across several key metrics:
- Identity Consistency: TPIGE achieved scores of 0.492 (Face-Cur) and 0.473 (Face-Arc), notably higher than the second-place team's scores of 0.467 and 0.441.
- Visual Quality: The method recorded the best FID (170) among top-tier participants, indicating superior image realism and fewer artifacts.
- User Preference: In a large-scale user study involving 3,000 comparisons, over 60% of participants preferred TPIGE's outputs over other state-of-the-art methods.
Discussion and Limitations
While TPIGE excels in identity preservation and visual quality, the authors noted a slight trade-off in CLIP scores (text alignment). Because the framework strictly enforces identity and image-prompt consistency, it may occasionally deviate from minor textual details to ensure the face remains recognizable. This highlights a fundamental tension in IPT2V research between literal prompt adherence and identity fidelity.
Conclusion
TPIGE demonstrates that high-quality, identity-preserving video generation does not strictly require expensive training or fine-tuning. By intelligently refining prompts, pre-aligning reference images, and applying spatiotemporal guidance during inference, TPIGE sets a new state-of-the-art benchmark. Future work will focus on improving text alignment and extending the framework to support multi-person scenarios.
Summary updated