A high-order graph convolutional network for homomorphic and heterogeneous skeletal motion retargeting

Jia Wei, Jun Li, Shujie Li, Yang Zhao, Hai MinJournal of Image and Graphics 2024 · DOI

Abstract

Skeletal motion retargeting aims to transfer motion data from a source character to a target character with a different skeletal structure. This task is challenging due to the high coupling between motion data and specific skeletal topologies. We propose a novel approach using a high-order graph convolutional network (GCN) and a feature separation strategy. By decoupling global body orientation from local pose data and employing high-order skeletal convolutions to capture long-range joint dependencies, our method significantly improves retargeting accuracy. Experimental results on the Mixamo dataset demonstrate a 38.6% improvement in accuracy for heterogeneous retargeting and a 74.8% improvement for homomorphic retargeting compared to current state-of-the-art methods.

Introduction

Motion retargeting is a cornerstone of modern animation, allowing developers to reuse motion capture (mocap) data across various characters. However, motion data is inherently "tightly coupled" with a character's specific skeletal structure. When source and target characters differ in proportions, joint counts, or movement ranges—such as retargeting real human motion to stylized virtual characters—existing "skeleton-aware" networks often struggle to separate the essence of an action from the skeleton it was recorded on. This leads to artifacts like unnatural poses, incorrect body orientations, or "foot sliding."

Methodology

Feature Separation and Orientation Decoupling

A major challenge in retargeting is the simultaneous learning of global orientation and local joint poses. We introduce a feature separation stage that extracts the character's body orientation as an independent feature. By isolating global rotation from local joint poses, we reduce the complexity of the learning task, allowing the network to focus on the nuances of the movement itself. This decoupling significantly stabilizes training and improves the naturalness of the retargeted results.

High-Order Skeletal Convolution

Standard GCNs typically aggregate information only from immediate (1-hop) neighbors in the skeletal graph. To capture broader structural context, we propose a high-order skeletal convolution operator. By utilizing powers of the adjacency matrix, the network can capture information from a wider neighborhood, learning relationships between distant joints (e.g., the hand and the shoulder) without a massive increase in parameters.

Network Architecture

The overall architecture is based on a pair of autoencoders ($G^A$ and $G^B$) that share a latent space. This ensures that the motion's semantic meaning is preserved even as it is decoded onto a different skeletal structure.

The proposed framework utilizes twin autoencoders with a shared latent space to bridge disparate skeletal structures.

Experiments and Results

We evaluated our method using two major benchmarks: Human3.6M (real human movements) and Mixamo (synthetic virtual characters). We specifically tested the "heterogeneous" scenario, retargeting 18-joint real-world data to virtual characters with higher joint counts (typically 22 or 28).

Quantitative Performance

Our method achieved significant improvements over the previous state-of-the-art (SAN):

  • Heterogeneous Retargeting: Accuracy improved by 38.6% on the Mixamo dataset.
  • Homomorphic Retargeting: For characters with the same joint count but different proportions, accuracy improved by 74.8%.

Qualitative Analysis

Visual results show that the proposed method maintains correct body orientation during complex turns and produces much more natural limb placements than previous techniques. By decoupling orientation, the model avoids common errors where characters fail to face the intended direction during dynamic movements.

Qualitative overview showing input motion (top) and the corresponding retargeting results (bottom) for a stylized character.

Conclusion

By combining orientation decoupling with high-order graph convolutions, we have developed a more robust framework for skeletal motion retargeting. Our approach effectively handles structural discrepancies between heterogeneous characters, resulting in more accurate and visually natural animations. Future work will explore integrating Inverse Kinematics (IK) as a constraint to further reduce minor artifacts like foot sliding.

Summary updated

Abstract

目的骨骼运动重定向是指将源角色的骨骼运动数据,修改后运用到另一个具有不同骨架结构的目标角色上,使得目标角色和源角色做出相同的动作。由于骨骼运动数据与骨架结构之间具有高耦合性,重定向算法需要从运动数据中分离出与骨架结构无关、只表示动作类型的特征。当源角色与目标角色骨架结构不同,且两者运动模式(如关节角变化范围)存在较大差异时,特征分离难度加大,重定向网络训练难度变大。针对该问题,提出了特征分离的方法和高阶骨骼卷积算子。方法在数据处理阶段,首先从运动数据中分离出一部分与骨架结构无关的特征,从而降低重定向网络训练难度,得到更好的重定向结果。另外,结合图卷积网络,本文针对人体骨架结构提出了高阶骨骼卷积算子。使用该算子,本文网络模型可以捕获更多有关骨架结构的信息,提高重定向结果的精度和视觉效果。结果在异构重定向任务中,本文方法在合成动画数据集Mixamo上与最新方法对比,重定向结果精度提升了38.6%。另外,本文方法也同样适用于同构重定向,结果精度比最新方法提升了74.8%。在从真人采集的运动数据到虚拟动画角色的异构重定向任务中,相比最新方法,本文方法能够明显减少重定向错误,重定向结果有更高的视觉质量。结论相比较于目前最新的方法,本文方法降低了特征分离的难度且更加充分挖掘了骨架的结构信息,使得重定向结果误差更低且动作更自然合理。
BibTeX
@article{wei2024high,
  author = {Jia Wei and Jun Li and Shujie Li and Yang Zhao and Hai Min},
  title = {{A high-order graph convolutional network for homomorphic and heterogeneous skeletal motion retargeting}},
  journal = {Journal of Image and Graphics},
  year = {2024},
  doi = {10.11834/jig.230909},
}
Figure