When animating a target character using the motion of a source actor, we encounter a fundamental geometric conflict: the morphology gap. A tall actor taking a stride moves their limbs across a specific spatial distance. If a generative model maps those exact spatial coordinates or raw joint trajectories onto a shorter character, the character's feet will slice through the floor, or their limbs will stretch unnaturally.
This issue, known as geometric proportion mismatch, represents a core challenge in cross-identity motion transfer. To build systems that can animate any character faithfully, we must understand why naive motion copying fails and how modern architectures decouple an actor's movement from their unique physical dimensions.
The Root Cause: Entangled Features and Shape Leakage
Why do standard deep learning models struggle to separate how an object moves from what the object looks like? The answer lies in how neural networks encode spatial data.
In video generation and diffusion models, motion and identity features are often deeply entangled. For example, Rinon Gal et al. demonstrated in Multi-Shot Character Consistency for Text-to-Video Generation that self-attention layers inherently conflate structural geometry with temporal motion patterns. When a network processes a driving video, its spatial queries capture the silhouette and limb lengths of the source actor alongside their actions.
When transferring this motion to a new target, this entanglement causes shape leakage. As highlighted by Yong Zhong et al. in PoseCrafter: One-Shot Personalized Video Synthesis Following Flexible Pose Control, forcing a target character to follow raw, unaligned source coordinates causes the target's body proportions to unnaturally warp, stretch, or compress to match the driving actor's physical build.
[Source Motion (Tall Performer)] ───> Naive Coordinate Mapping ───> [Target Character (Short)]
│
▼
• Foot Sliding
• Mesh Distortion
• Identity "Melting"
This breakdown is even more severe when the characters have different physical volumes or structural constraints:
- Self-Penetration: Duygu Ceylan observed in Contact-Aware Retargeting of Skinned Motion that skeleton-only retargeting ignores the physical volume of the character mesh. Without accounting for body thickness, a hand gesture that rests on a heavy-set actor's hip will plunge directly into the mesh of a slender target character.
- Semantic Clipping: For fine-grained movements like hand gestures, Zijie Ye et al. noted in Semantics2Hands: Transferring Hand Motion Semantics between Avatars that direct joint mapping causes fingers to clip through surfaces. Because the spatial position of an end-effector (like a fingertip) depends entirely on specific bone lengths, a "pinch" gesture can easily become a physical impossibility on a hand with different proportions.
Strategy 1: Relative and Residual Transformations
To prevent shape leakage, models must shift from absolute coordinate tracking to invariant motion representations. One of the most effective ways to achieve this is through relative joint angles and part-aware residuals.
Instead of telling a model where a hand is in 3D space, relative transfer focuses on how the joints rotate relative to their parent bones. As shown by Borun Xu et al. in Motion and Appearance Adaptation for Cross-domain Motion Transfer, prioritizing joint angles over absolute spatial coordinates prevents the geometric warping that occurs when a driver's specific proportions are forced onto a target image. An elbow bending 90 degrees remains a 90-degree bend, whether the arm is 50 centimeters or one meter long.
Absolute Mapping (Fails): Source Hand at (X, Y, Z) ───> Forces Target Hand to (X, Y, Z) [Stretches Mesh]
Relative Mapping (Works): Source Elbow rotates θ° ───> Target Elbow rotates θ° [Preserves Scale]
In the facial domain, a similar principle applies. Directly copying facial landmark coordinates from a human driver to a stylized avatar causes severe distortion. To solve this, Zhiyuan Chen et al. implemented part-aware residual transformations in EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions. Instead of transferring absolute positions, the model calculates the displacement of a feature (e.g., how much the lip moves relative to the jaw) and applies that delta to the target's unique facial geometry, preserving the target's identity across diverse artistic styles.
Strategy 2: Explicit 3D Structural Alignment
When animating 2D images or complex meshes, relative angles alone may not suffice; the model needs an explicit understanding of the target's 3D structure before applying motion.
Shenhao Zhu et al. tackled this in Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance by introducing 3D parametric models (such as SMPL) as an intermediary layer. By explicitly mapping the reference subject's unique 3D body shape and scaling the driving motion sequence to fit that specific parametric structure, the latent diffusion model generates animations that respect the original subject's dimensions, preventing the character from unnaturally morphing into the driver's shape.
[Driving Video Motion] ───┐
▼
[3D Parametric Model] ───> Normalizes & Scales Motion ───> [Latent Diffusion Generation]
▲
[Target Image Geometry] ──┘
For skeleton-free characters—such as monsters, ghosts, or stylized art—traditional joint-based scaling breaks down entirely. In TapMo: Shape-aware Motion Generation of Skeleton-free Characters, Jiaxu Zhang et al. utilized shape-aware diffusion models that analyze the target character's surface mesh geometry directly. By predicting adaptive mesh handles based on the character's unique topology, the model ensures that trajectories do not cause unnatural mesh distortions, such as collapsing or tearing during complex movements.
Strategy 3: Physics and Dynamic Constraints
Even with perfect geometric scaling, a motion can look unconvincing if it violates basic physics. A character with a shifted center of mass or heavier body proportions cannot move exactly like a lightweight actor without looking unstable.
To enforce realism, modern architectures incorporate differentiable physics engines and stability constraints directly into the motion loop. Omid Taheri et al. demonstrated this in HUMOS: Human Motion Model Conditioned on Body Shape. By conditioning a generative transformer on explicit body shape parameters and passing the output through a physics-informed layer, the network learns to adjust foot placement and balance dynamically. This optimization eliminates artifacts like foot sliding and ground penetration by ensuring the motion satisfies the physical constraints of the target's specific mass distribution.
When the morphology gap is extreme—such as controlling a non-human avatar using sparse human VR inputs—Daniele Reda et al. showed in Physics-based Motion Retargeting from Sparse Inputs that combining reinforcement learning with a physics simulator allows the system to translate human intent into functional, balanced movements for characters with completely different limb lengths and centers of mass.
By decoupling motion from structure through relative transformations, anchoring animations with 3D parametric models, and enforcing physical constraints, modern motion transfer frameworks successfully bridge the morphology gap—ensuring that characters retain their unique identity and physical plausibility, no matter who is driving the motion.