Traditional motion retargeting relies on a comforting assumption: the source character and the target character share the same underlying skeletal structure. If both characters possess the exact same joint count and hierarchical connectivity, transferring a motion is primarily a matter of adjusting for variations in bone lengths.
However, this assumption breaks down entirely when animating across different species, robotic platforms, or stylized creatures. Attempting to map a bipedal human's walk cycle to a quadrupedal dog, a multi-limbed monster, or a mechanical vehicle introduces a fundamental structural mismatch. As noted by Jia Wei et al. in A high-order graph convolutional network for homomorphic and heterogeneous skeletal motion retargeting, standard joint-to-joint mapping techniques fail in these heterogeneous scenarios because there is no direct one-to-one correspondence between the source and target joints. When skeletal topologies diverge, rigid geometric mappings yield unnatural distortions, joint clipping, or a complete loss of the motion's semantic intent.
To bridge this topological gap, modern motion transfer frameworks abstract movement away from rigid joint hierarchies. They achieve this through three core strategies: tokenizing skeletal structures, mapping high-level semantic body parts, and bypassing skeletons entirely via rig-free geometric deformation.
Abstracting Skeletons into Flexible Tokens
To handle characters with varying joint counts, recent architectures move away from rigid graph structures and instead treat joints as collections of independent features. By transforming a skeleton into a set of discrete tokens, models can leverage the permutation-invariant and flexible nature of attention mechanisms.
In Skeleton-Aware Motion Retargeting Using Masked Pose Modeling, Giulia Martinelli et al. implement a transformer framework that treats individual joints as tokens within a sequence. By utilizing a masked pose modeling strategy—similar to how large language models predict missing words—the network learns a deep, generalized understanding of motion dynamics that is decoupled from any specific bone hierarchy. This tokenized perspective allows a single model to generalize across scales, proportions, and entirely different topologies without requiring architectural modifications for each character pair.
Traditional Graph Mapping:
[Source Joint i] ---> (Fails if no matching target joint exists) ---> [Target Joint i]
Tokenized Attention Mapping:
[Joint Tokens] ---> [Spatial-Temporal Transformer] ---> [Target Tokens]
(Variable Count) (Learns latent semantic correlations) (Variable Count)
This strategy of cross-topology interoperability is reinforced by Junkun Jiang et al. in A Dual-Masked Auto-Encoder for Robust Motion Capture with Spatial-Temporal Skeletal Token Completion. The authors demonstrate that by applying spatial and temporal positional encodings to joint tokens, a dual-masked autoencoder can map correlations between entirely different skeletal definitions. This allows motion knowledge captured in one standard format to seamlessly translate to another, resolving structural discrepancies during the data-fusion phase.
Mapping Motion by Semantic Body Parts
An alternative intuition for overcoming topological constraints is to change the granularity of the transfer. Instead of mapping motion at the level of individual joints, frameworks can operate at the level of functional body parts. While a human and a quadruped animal do not share identical joint hierarchies, they do share semantic components—such as limbs that support weight or execute coordinated steps.
Lei Hu et al. formalize this approach in Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part. The authors introduce a Pose-aware Attention Network (PAN) that treats distinct body parts as the fundamental units of motion transfer. Rather than forcing a rigid mathematical alignment between a human arm and a dog's front leg, PAN dynamically weights the importance of different joints based on the active pose. This allows the network to capture the behavioral essence of an action and distribute it appropriately across the target's unique structural configuration.
Similarly, when transferring behaviors across diverse animal species, models must account for distinct anatomical habits. In Behave Your Motion: Habit-preserved Cross-category Animal Motion Transfer, Zhimin Zhang et al. combine a vector-quantized variational autoencoder (VQ-VAE) with category-specific habit encoders. By isolating the underlying intent of a motion from the species-specific mechanics dictated by anatomy, the framework successfully animates unobserved target species while preserving their natural physiological constraints.
Bypassing Skeletons with Rig-Free Transfer
The most radical solution to the topology problem is to eliminate the skeletal rig altogether. Animation rigs and controllers are traditionally bound to specific character meshes, making cross-domain transfer highly labor-intensive. By mapping motion directly in a dense geometric or visual space, researchers can animate objects that lack any shared structural lineage.
In Skeleton-Free Pose Transfer for Stylized 3D Characters, Jimei Yang et al. introduce a unified, part-based articulation model that operates directly on raw 3D mesh data. By abandoning hierarchical skeletons, this framework can transfer complex poses to stylized monsters, animals, or even inanimate objects that feature extra limbs, tails, or non-humanoid structures.
Rig-Based Pipeline (Topology-Constrained):
Source Video -> Skeletal Fitting -> Joint Rotation Mapping -> Target Rig -> Target Mesh
Rig-Free Pipeline (Topology-Agnostic):
Source Video -> Semantic/Geometric Embedding -> Dense 3D Deformation -> Target Mesh
This concept scales to open-domain scenarios where motion is lifted directly from video source material. For instance, Yarin Bekor et al. develop a rig-free framework in Gaussian See, Gaussian Do: Semantic 3D Motion Transfer from Multiview Video. By projecting 2D video diffusion priors into a view-aware 3D space, they achieve semantic motion transfer across disparate categories—such as translating the rearing motion of a horse into the lifting of a sports car's front wheels. Because the model operates on semantic intent rather than joint coordinates, it bypasses the topological limitations that cause traditional rigging pipelines to fail.
Preserving Physical and Geometric Plausibility
When mapping motion between highly divergent structures, the primary risk is the loss of physical realism. A valid pose on a human skeleton can easily cause self-intersection (clipping) or floating limbs when mapped onto a character with a wider torso or shorter legs.
To enforce structural validity without relying on paired training data, early unsupervised networks introduced explicit kinematic constraints. In Neural Kinematic Networks for Unsupervised Motion Retargetting, Jimei Yang et al. combine a differentiable Forward Kinematics (FK) layer with an adversarial cycle-consistency loss. The FK layer ensures that the network's internal representations map directly to valid geometric transformations, preventing the generation of physically impossible joint configurations.
As characters interact with themselves or their surroundings, preserving contact points becomes crucial. Ruben Villegas et al. address this in Contact-Aware Retargeting of Skinned Motion by pairing a geometry-aware Recurrent Neural Network (RNN) with an encoder-space optimization strategy. This combination ensures that critical self-contacts (such as a hand touching the face) are maintained across characters of varying shapes and sizes.
When structural variations become extreme—such as characters with missing or additional limbs—dense interaction perception is required. Zijie Ye et al. demonstrate in Skinned Motion Retargeting with Dense Geometric Interaction Perception that modeling the dense spatial relationship between a character's surface meshes prevents interpenetration and preserves the original context of the motion. Furthermore, as shown by Baiyi Li in Two-Person Interaction Augmentation with Skeleton Priors, incorporating explicit skeleton priors and joint probability factorizations allows models to preserve complex, contact-rich interactions even when scaling movements between characters with vastly different body proportions.
Finally, to ensure smooth transitions between these disparate structural spaces, recent methods model motion as a continuous trajectory. In MoReFlow: Motion Retargeting Learning through Unsupervised Flow Matching, Wontaek Kim et al. utilize conditional flow matching to align discrete motion tokens. By treating retargeting as a continuous vector field transformation, the model establishes a reversible, fluid mapping between highly divergent topologies, such as a bipedal human and a quadrupedal robot, ensuring that the resulting animations remain stable, natural, and free of jitter.
Key Foundational Work
Neural Kinematic Networks for Unsupervised Motion Retargetting (2018) — contributed 1 concept elements to this direction.
Real‐time locomotion control by sensing gloves (2006) — contributed 1 concept elements to this direction.