Part Level Decomposition for Human Motion Generation

17 concepts12 papers

When a human is asked to "walk forward while waving their left hand and nodding," our brains do not treat this as a single, indivisible command. Instead, we distribute the tasks: our legs handle locomotion, our left arm executes the wave, and our neck controls the nod.

Traditional text-to-motion models, however, treat the human body as a monolithic entity. They attempt to map a global text prompt directly to an entire 3D skeleton. This creates a severe bottleneck. When a single neural network tries to resolve a complex sentence into full-body joint angles, it frequently suffers from semantic leakage—a phenomenon where the model confuses left and right limbs, blends distinct actions into an unnatural mess, or completely ignores localized instructions because a dominant action (like walking) overpowers the model's attention.

To solve this, a powerful paradigm shift has emerged: decomposing both the text prompt and the human skeleton into fine-grained, part-level components before attempting any alignment.

Anatomical Partitioning and Localized Narratives

The first step in this approach is breaking down the physical body and the text into corresponding anatomical regions. Instead of feeding a model a single global sentence, the text is split into independent instructions for the arms, legs, and torso.

In LGTM: Local-to-Global Text-Driven Human Motion Diffusion Model, Haowen Sun leverages a Large Language Model (LLM) to dissect complex text prompts into "localized narratives" targeted at specific body parts. This partitioning ensures that specific motion commands map directly to the relevant joints, preventing semantic interference across different body parts.

[Global Prompt]: "Walk forward while waving with the right hand"
                       │
                       ▼ (LLM Decomposition)
┌──────────────────────┴──────────────────────┐
▼                                             ▼
[Local Narrative A]: "Walk forward"           [Local Narrative B]: "Wave right hand"
▼                                             ▼
[Target Joints]: Lower Body                   [Target Joints]: Right Arm

Similarly, in Motion-example-controlled Co-speech Gesture Generation Leveraging Large Language Models, Bohong Chen et al. tokenize motion data using an anatomically partitioned approach that separates the upper body, lower body, and hands. By treating these regions as distinct streams of tokens, the model gains granular control over individual body parts, allowing for highly customizable and precise animation.

Hierarchical Text-Motion Alignment

Why does this part-level approach perform so much better? The answer lies in how language naturally maps to physical movement. High-level verbs like "run" imply full-body coordination, while verbs like "scratch" or "point" are highly localized.

To capture this multi-granular reality, modern architectures implement hierarchical text-motion alignment. In Quo Vadis, Motion Generation? From Large Language Models to Large Motion Models, Sipeng Zheng et al. introduce a multi-level text strategy that pairs high-level body summaries with granular, part-level descriptions. This dual-level supervision allows the model to learn both the forest and the trees: it understands the global context of the action while precisely mapping specific linguistic commands to localized joint movements.

This concept is taken a step further in KinMo: Kinematic-aware Human Motion Understanding and Generation, which introduces a staged refinement process. The model progressively encodes text and motion embeddings by starting with global actions, moving to group-level descriptions (e.g., "left arm"), and finally incorporating interaction-level descriptions (e.g., "torso and left arm interaction"). By structuring the alignment hierarchically, the model learns the subtle dependencies between parts without sacrificing localized precision.

This granular mapping is heavily supported by modern datasets. For example, Inter-X: Towards Versatile Human-Human Interaction Analysis, developed by Liang Xu et al., provides multi-level textual descriptions detailing specific body part movements, explicitly linking specific verbs to localized physical articulations.

Compositional Generalization and Part Dynamics

Beyond preventing semantic leakage, part-level decomposition unlocks compositional generalization. In traditional datasets, a model might never see the exact prompt "skipping while playing the violin." If it treats the prompt monolithically, it will fail.

However, if the model can decouple and recombine movements, it can retrieve or generate the lower-body motion for "skipping" and the upper-body motion for "playing the violin" independently. In Scaling Large Motion Models with Million-Level Human Motions, Ye Wang et al. demonstrate that training motion models with hierarchical descriptions specifying actions for distinct body regions enhances the model's ability to decouple and recombine movements. This allows systems to successfully execute complex, multi-part instructions that global descriptions typically fail to capture.

Furthermore, different body segments operate on entirely different temporal scales and kinematic constraints. The fine-grained, rapid movements of a violinist's fingers require a different level of attention than the slower, sweeping movements of their torso. As shown in the framework introduced by the authors of SyncViolinist: Music-Oriented Violin Motion Generation Based on Bowing and Fingering, separating motion generation by body parts is essential because different segments possess distinct temporal characteristics.

By recognizing these localized differences, models can optimize their generation strategies per limb. For instance, MoRAG - Multi-Fusion Retrieval Augmented Generation for Human Motion breaks down full-body generation into independent, part-specific retrieval tasks (hands, torso, legs). This allows the system to handle localized actions and varied language far better than monolithic retrieval methods.

The Divide-and-Unite Mechanism

If body parts are generated or retrieved independently, how do we prevent the final animation from looking like a disjointed collection of limbs? The secret to making this architecture work is a "divide-and-unite" or coordination phase.

In Combo: Co-speech holistic 3D human motion generation and efficient customizable adaptation in harmony, Chao Xu et al. propose a strategy that explicitly separates facial and body movement learning before unifying them. This structural separation followed by a unification layer leads to highly synchronized and coordinated full-body motions compared to purely end-to-end approaches.

┌────────────────────────────────────────────────────────┐
│                  Divide-and-Unite                      │
├───────────────────────────┬────────────────────────────┤
│  Part-Specific Generation │ Global Coordination Layer  │
├───────────────────────────┼────────────────────────────┤
│ • Independent dynamics    │ • Resolves joint blending  │
│ • Prevents limb blending  │ • Ensures smooth weight    │
│ • High local fidelity     │   transfers and balance    │
└───────────────────────────┴────────────────────────────┘

To bridge the gap between individual part dynamics and global motion coherence in real-time, frameworks like SPORT: From Zero-shot Prompts to Real-time Motion Generation by Bin Ji et al. decompose motion control and text-motion alignment to the body-part level, significantly improving the generation of nuanced and controllable animations.

To ensure these parts fuse seamlessly without losing their distinct properties, the authors of MoSCo: Real-time and Efficient Text-to-Motion Synthesis via Delta Training employ a part-aware coordinator. They demonstrate that decomposing motion into parts and then coordinating them effectively bridges the gap between individual part dynamics and global motion coherence. Finally, SegMo: Segment-aligned Text to 3D Human Motion Generation ensures this alignment happens across time as well, decomposing both text and motion into semantically coherent temporal segments to achieve precise local-to-global alignment.

By moving away from monolithic generation and embracing part-level decomposition, modern motion models mirror the hierarchical nature of human motor control. They prevent semantic leakage by isolating textual commands to specific joints, achieve compositional flexibility by mixing and matching independent limb actions, and maintain naturalism through dedicated global coordination layers.

Key Foundational Work

MoRAG - Multi-Fusion Retrieval Augmented Generation for Human Motion (2025) — contributed 2 concept elements to this direction.
Quo Vadis, Motion Generation? From Large Language Models to Large Motion Models (2024) — contributed 2 concept elements to this direction.
Scaling Large Motion Models with Million-Level Human Motions (2025) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_text_motion_alignment_e4cc4235,
  title = {hierarchical text-motion alignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-text-motion-alignment/rdr_e4cc42358346423a818ac99ece9c394a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_hierarchical_text_motion_alignment_e4cc4235_v1,
  title = {hierarchical text-motion alignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-text-motion-alignment/rdr_e4cc42358346423a818ac99ece9c394a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake