TL;DR
DiVA (Digital Life Simulator) introduces a novel framework that combines a Multimodal Large Language Model (MLLM) router with a decoupled, three-part video generation pipeline to enable long-term, open-ended interactive character simulations. By introducing an Anchored Video Continuation (AVC) module that periodically resets the character to stable, high-quality "anchor states," DiVA eliminates the cumulative error and identity drift that plague traditional long-video generation models while unlocking highly dynamic pose changes.
What Problem Does This Paper Address?
Creating interactive, long-term digital character experiences—termed Digital Life Simulation (DLS)—presents a fundamental stability-expressiveness trade-off. Current end-to-end long-video generation models suffer from severe quality degradation and identity drift over multi-turn interactions because minor errors accumulate autoregressively over time.
On the other hand, audio-driven avatar models excel at local identity preservation and lip-syncing, but their strict coupling to audio constraints prevents them from executing large, macroscopic physical transitions (such as moving from sitting to standing or dramatic camera shifts). DiVA aims to bridge this gap, delivering an infinite, continuous interaction loop with high visual fidelity, precise user-intent tracking, and rich motion dynamics.
Methods
DiVA replaces the standard monolithic video generation approach with a decoupled, stateful pipeline orchestrated by an MLLM router (such as GPT-4 or Qwen).
The system operates in three distinct, coupled phases:
- Waiting Video: Generates subtle, loop-compatible idle motions (e.g., breathing, blinking) using an Image-to-Video (I2V) model fine-tuned via Low-Rank Adaptation (LoRA) to prevent a frozen avatar feel.
- Action Video: Triggered by user input, this module aligns text and text-to-speech (TTS) audio (using CosyVoice) with an audio-text conditional video model to render speech-aligned gestures and expressions.
- Anchored Video Continuation (AVC): Instead of standard frame interpolation, the AVC module conditions on the final 25% of frames from the Action Video and seamlessly transitions the character back to a pre-defined, high-quality anchor state (e.g., a specific sitting or standing pose). This phase is intentionally decoupled from audio alignment, allowing the model to execute large-scale pose and camera movements smoothly.
Additionally, DiVA introduces a Visual-Referential Grounding mechanism for spatial click interactions. Instead of feeding raw numerical coordinates to the MLLM, the system dynamically draws a visual marker (a red dot) on the frame, allowing the MLLM to leverage its native visual referring capabilities to infer user intent.
Datasets
The authors curated two primary in-house datasets to train and specialize their pipeline components:
- AVC Training Dataset: 45k video-text pairs containing meaningful changes in body pose, camera distance, or viewpoint, annotated using Qwen2.5-VL.
- Waiting Video Dataset: 3k clips featuring low-amplitude, loop-compatible motions like minor head and hair movements.
For evaluation, the authors established a long-horizon benchmark consisting of 60 multi-turn character simulations spanning human, anime, stylized, and animal identities, alongside 50 interaction pairs for evaluating spatial click grounding.
Key Insights and Ideas
- The Power of Explicit Resets: Traditional single-stream video continuation models allow errors to emerge implicitly from recent latent history. DiVA's "aha moment" is realizing that periodically forcing the character back to a vetted, high-quality anchor state completely halts error accumulation and identity drift.
- Decoupling Audio and Motion: By separating the speech-aligned action phase from the macro-transition phase (AVC), the model bypasses the motion-restricting constraints of lip-syncing models, unlocking creative and expansive dynamic effects.
- Visual Markers over Coordinates: Transforming a difficult numerical-to-visual grounding problem into a simple visual referring task by drawing a dot on the image significantly boosts the MLLM's spatial reasoning accuracy without requiring extra video-model training.
Key Results
DiVA was benchmarked against state-of-the-art long-video and audio-avatar models (including FramePack, InfiniteTalk, and LongCat-Video):
- Anti-Drift Performance: DiVA reduced Quality Drift to 0.0341, compared to InfiniteTalk's 0.0947 (a ~64% relative reduction), demonstrating exceptional long-term visual stability.
- Dynamic Range: DiVA achieved the highest Dynamic Degree (0.4833) among all tested models, proving that its quality stability does not come at the expense of motion.
- Inference Efficiency: Under its real-time causal configuration, DiVA achieved an end-to-end response latency of ~1.5 seconds and an interactive rendering throughput of ~15 FPS on NVIDIA H800 GPUs, vastly outperforming baselines like FramePack (28.5s per turn).
- Human Evaluation: In a user study evaluating transition naturalness, DiVA was preferred over the strong Wan FLF2V baseline in 76.7% of comparisons for Transition Fidelity and 71.7% for Physical Plausibility.
Limitations and Future Work
Despite its strong performance, DiVA inherits fine-grained generation failures from its underlying video backbones. When a selected target anchor differs drastically from the preceding action segment, the AVC transition can occasionally look like a "cross-fade" rather than a physically plausible motion.
Furthermore, the model is susceptible to visual hallucinations—generating illogical artifacts that lack physical grounding (such as objects appearing out of thin air or defying common sense). The authors note that balancing creative prompt-following with strict physical realism remains an open direction for future research.
Summary updated