Diffusion Models for Human Motion Synthesis

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 119 papers21 conceptsfrozen 2026-10-04

Generating realistic human motion is a high-dimensional balancing act. A model must ensure that joints follow anatomical constraints, movements align with external cues like music or text, and the resulting sequence remains temporally smooth without collapsing into a generic "average" pose.

While early approaches relied on Variational Autoencoders (VAEs) or Generative Adversarial Networks (GANs), a survey by Wentao Zhu and colleagues highlights a massive paradigm shift toward Denoising Diffusion Models. By treating motion generation as the iterative removal of noise from a random skeletal sequence, researchers are achieving unprecedented levels of realism and diversity across tasks ranging from sign language synthesis to group choreography.

Sculpting Motion from Noise

The core intuition behind these models is that human motion can be "sculpted." Instead of trying to map a text prompt directly to a 3D coordinate in one step—a process that often leads to jittery or "stiff" results—diffusion models start with pure Gaussian noise.

In the LaserHuman framework by Peishan Cong and Zhiyang Dou, this process is guided by language and scene geometry. The model iteratively refines the noise, ensuring the "person" in the animation doesn't just walk, but walks specifically up the stairs defined in the 3D environment. This iterative refinement allows the model to satisfy complex spatial constraints that deterministic models often ignore.

Bridging the Gap Between Discrete States

One of the most difficult problems in animation is the "in-betweening" task: how do you move naturally from Pose A to Pose B? This is particularly vital in sign language, where jerky transitions can change the meaning of a sentence.

Shengeng Tang and Jiayi He addressed this with Sign-D2C. They framed transition generation as a "mask-and-recover" task. By conditioning a diffusion model on the latent representations of the preceding and succeeding frames, the system uses cross-attention to fill the gap with fluid, contextually accurate motion. Because the diffusion process is probabilistic, it can capture the non-linear, expressive "sway" of human movement that simple interpolation misses.

Synthesizing Full Bodies from Sparse Data

In VR and motion capture, we often have very little data to work with—perhaps just the position of a headset and two controllers. Angela Castillo and Ali Thabet developed BoDiffusion to solve this "sparse-to-full" problem.

By conditioning the denoising steps on these sparse signals, the model learns the statistical likelihood of where the rest of the body should be. This approach significantly reduces "foot skating"—the immersion-breaking glitch where a character's feet slide across the floor—because the diffusion process naturally gravitates toward physically plausible joint configurations.

Scaling to Interactions and Groups

The complexity of motion synthesis increases exponentially when multiple people are involved. Researchers are now moving beyond single-actor models to handle "cross-talk" between skeletons:

  • Interactions: Baptiste Chopin and Hao Tang 0005 introduced BiGraphDiff, which uses a bipartite graph module within a diffusion transformer. This allows the model to learn how the movement of one person (e.g., a dancer being lifted) directly influences the noise-reduction process of the other.
  • Group Dynamics: For large-scale choreography, Cunjian Chen and Jun Liu proposed ST-GDance. This model decouples spatial and temporal modeling, using Graph Convolutional Networks (GCNs) to prevent dancer collisions while the diffusion backbone ensures everyone stays in sync with the music's rhythm.

Beyond Transformers: The Mamba Evolution

While Transformers are the standard backbone for these models, they struggle with the long-range temporal dependencies required for long animations. Ziyun Qian and colleagues recently introduced SMCD, which replaces traditional attention mechanisms with the Mamba (State Space Model) architecture. By integrating Mamba into the diffusion denoiser, they achieved more coherent motion style transfer, capturing subtle stylistic details like gait patterns over much longer sequences than previously possible.

This shift toward diffusion-based refinement—whether applied to hand forecasting in egocentric video by Masashi Hatano or fine-grained dance generation in FineNet—represents a fundamental change in how machines understand human movement: not as a fixed path to be predicted, but as a structured distribution to be sampled and refined.

Papers in this version

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_diffusion_models_for_human_motion_synthe_222d4aee,
  title = {Diffusion Models for Human Motion Synthesis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/diffusion-models-for-human-motion-synthesis/rdr_222d4aeeac3146f986c4f0497eaacbc3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_diffusion_models_for_human_motion_synthe_222d4aee_v1,
  title = {Diffusion Models for Human Motion Synthesis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/diffusion-models-for-human-motion-synthesis/rdr_222d4aeeac3146f986c4f0497eaacbc3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}