Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance

Li Hu, Guangyuan Wang, Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang, Bang Zhang, Liefeng Bo

Abstract

Animate Anyone 2 introduces "environment affordance" to character animation, enabling AI-generated characters to interact realistically with their surroundings. By utilizing shape-agnostic masking, a dedicated object guider, and depth-aware pose modulation, the framework achieves state-of-the-art integration between animated subjects and complex backgrounds, overcoming common issues like human-object interaction clipping and spatial inconsistency.

Introduction

While recent diffusion-based models excel at maintaining character appearance consistency during motion, they often treat characters and backgrounds as separate layers, resulting in a "sticker-on-video" effect. Animate Anyone 2 addresses this lack of environment affordance, focusing on three primary challenges: Human-Object Interaction (HOI), spatial consistency in complex movements, and preventing shape leakage from the driving video onto the target character.

Methodology

The framework builds on the original Animate Anyone architecture (ReferenceNet + Denoising 3D UNet) but introduces three major innovations to enhance environmental interaction.

The Animate Anyone 2 framework introduces environment conditioning alongside motion to ensure characters interact naturally with their surroundings.
  • Environment Formulation & Shape-Agnostic Masks: The environment is provided as a conditional input. To prevent the model from simply memorizing the source person's silhouette, "shape-agnostic masks" create loose boundaries, forcing the model to learn the relationship between the character and the scene rather than just filling a specific shape.
  • Object Injection via Spatial Blending: For scenes involving specific objects (like a basketball or a bicycle), a lightweight Object Guider extracts multi-scale features. These are injected into the denoising process via Spatial Blending to ensure interaction dynamics are preserved.
  • Depth-wise Pose Modulation: Standard 2D skeletons are augmented with structured depth maps, providing a nuanced understanding of limb-to-limb spatial relationships without the risks of shape leakage associated with raw depth maps.

Datasets and Evaluation

The researchers utilized a massive new internal collection of 100,000 character videos featuring diverse scene types and complex interactions. Benchmarking was performed on the standard TikTok dataset and a real-world test set of 100 videos to evaluate generalizability.

Key Results

Animate Anyone 2 sets a new state-of-the-art (SOTA) across multiple metrics. On the TikTok benchmark, it achieved an FVD (Frechet Video Distance) of 144.65, significantly lower than the original Animate Anyone (171.90).

Qualitative results demonstrate the model's ability to handle complex interactions, such as a character walking on a treadmill without sliding or clipping.

Qualitative evaluations show superior performance in scenarios involving complex physics, such as riding horses, playing instruments, or using gym equipment. In these cases, previous models typically showed significant "sliding" or limb distortion, whereas Animate Anyone 2 maintains structural integrity and environmental grounding.

Conclusion and Future Work

By shifting from character-centric to interaction-centric animation, Animate Anyone 2 significantly advances the realism of AI-generated video. Despite this progress, challenges remain in scaling to very small objects and handling extreme proportion differences between the source and target characters. Future work will focus on refining intricate hand-object interactions and improving robustness for diverse character retargeting.

Summary updated

Abstract

Recent character image animation methods based on diffusion models, such as Animate Anyone, have made significant progress in generating consistent and generalizable character animations. However, these approaches fail to produce reasonable associations between characters and their environments. To address this limitation, we introduce Animate Anyone 2, aiming to animate characters with environment affordance. Beyond extracting motion signals from source video, we additionally capture environmental representations as conditional inputs. The environment is formulated as the region with the exclusion of characters and our model generates characters to populate these regions while maintaining coherence with the environmental context. We propose a shape-agnostic mask strategy that more effectively characterizes the relationship between character and environment. Furthermore, to enhance the fidelity of object interactions, we leverage an object guider to extract features of interacting objects and employ spatial blending for feature injection. We also introduce a pose modulation strategy that enables the model to handle more diverse motion patterns. Experimental results demonstrate the superior performance of the proposed method.
BibTeX
@inproceedings{hu2025animate,
  author = {Li Hu and Guangyuan Wang and Zhen Shen and Xin Gao and Dechao Meng and Lian Zhuo and Peng Zhang and Bang Zhang and Liefeng Bo},
  title = {{Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2025},
}
Figure