Overcoming Mode Collapse in Generative Motion Synthesis

37 concepts37 papers1 proposal

When a human is asked to "walk across the room," they might do so with a brisk stride, a casual saunter, or a cautious shuffle. However, traditional deep learning models for motion synthesis often struggle to capture this variety. Instead of producing a spectrum of plausible movements, these models frequently suffer from mode collapse—a phenomenon where the generator converges on a single, "safe" output, or a mean pose that represents a statistical average of all possible movements.

The result is motion that feels robotic, repetitive, or unnaturally static. Researchers are currently shifting away from deterministic architectures toward probabilistic frameworks to restore the natural stochasticity of human behavior.

The Problem of the Mean Pose

The core challenge in motion synthesis is that the mapping from a command (like "dance") to a sequence of joint angles is one-to-many. As Ghorbani et al. observe, humans rarely perform the same action identically twice.

When models are trained using standard regression losses (like Mean Squared Error), they are mathematically incentivized to minimize the distance to all possible ground-truth examples. If the training data contains ten different ways to jump, the model may settle on the average of those ten trajectories. Kundu et al. and Cai et al. argue that this "mean pose" problem is the primary driver of robotic and lifeless animations. In extreme cases, such as subtle or slow movements, Han et al. note that models may default to generating static-looking frames entirely.

Why GANs Struggle with Diversity

Generative Adversarial Networks (GANs) were an early attempt to solve this by using a discriminator to penalize "average" poses. However, GANs are notoriously prone to mode collapse in the motion domain. Zhu et al. and Yang et al. point out that GANs often ignore random noise seeds, producing the same "safe" gesture regardless of the input.

This lack of diversity is particularly problematic in specialized domains:

  • Physics-based Control: Fidler notes that adversarial imitation learning often learns only the simplest motions, ignoring the complex behaviors in a dataset.
  • Interaction: Liu et al. show that adversarial motion priors can converge on a narrow set of stable movements, preventing varied stylistic expressions.
  • Music-Driven Motion: Zhao et al. find that GAN-based orchestral conducting models fail to capture the full variety of human performance, leading to repetitive baton movements.

The Probabilistic Shift: Flows and Diffusion

To move beyond the mean pose, researchers are adopting models that explicitly define a probability distribution over possible motions.

Normalizing Flows and VAEs

One approach is to map motion into a structured latent space. Yin et al. utilize Normalizing Flows to represent the diverse range of natural movements as a density, allowing for more stable reconstruction than previous methods. Similarly, Wen et al. use generative flows to ensure that stylized motions (like an "angry" walk) retain the inherent stochasticity of human movement rather than becoming a fixed, repetitive loop.

The Diffusion Revolution

Diffusion models have recently emerged as a powerful solution to mode collapse. By learning to reverse a gradual noising process, models like the Human Motion Diffusion Model (MDM) by Tevet et al. can sample diverse sequences from the same prompt. Unlike GANs, diffusion models are more stable and better at covering the entire distribution of the training data. Yin et al. demonstrate that autoregressive diffusion models can generate high-quality, diverse motions even when the input data is noisy or incomplete.

Steering Diversity with Guidance

Even with probabilistic models, there is a risk of "redundant" outputs where the model samples very similar trajectories. To address this, researchers are developing methods to "steer" the generation process toward distinct modes.

Pandey et al. introduced "guidance energies" that can be applied to pre-trained diffusion models to discover multiple, physically distinct ways an object might move (e.g., a drawer opening vs. closing). In the realm of speech, Meng et al. propose decoupling a person's unique movement style from the speech content. By using stylistic cues from reference videos, they generate gestures that are synchronized with audio but rich in individual personality and variation.

Conclusion

The transition from deterministic to probabilistic modeling represents a fundamental shift in how we teach machines to move. By moving away from the "average" and embracing the full distribution of human behavior, these new frameworks—ranging from hierarchical latent models (Ghorbani et al.) to diffusion-based in-betweening (Tevet et al.)—are enabling characters that no longer just "perform an action," but do so with the nuance and unpredictability of real life.

Key Foundational Work

Long-Term Human Video Generation of Multiple Futures Using Poses (2020) — contributed 1 concept elements to this direction.
BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN (2019) — contributed 1 concept elements to this direction.
Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_mode_collapse_in_motion_synthesis_22080585,
  title = {mode collapse in motion synthesis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mode-collapse-in-motion-synthesis/rdr_2208058539534b34a9b6848a05b2e291},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_mode_collapse_in_motion_synthesis_22080585_v1,
  title = {mode collapse in motion synthesis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mode-collapse-in-motion-synthesis/rdr_2208058539534b34a9b6848a05b2e291/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake