Introduction
While current text-to-motion diffusion models excel at generating the semantic content of an action (the "what"), they often struggle to capture the expressive quality (the "how"). For instance, a request for a "heavy, sluggish jump" versus a "light, airy jump" frequently results in nearly identical motions. This "expressive gap" stems from the scarcity of high-quality motion-text datasets and the inherent ambiguity of natural language when describing subtle physical nuances.
To bridge this gap, LaMoGen (Laban Movement-Guided Diffusion) introduces a zero-shot, inference-time optimization method. By leveraging Laban Movement Analysis (LMA)—a comprehensive framework used by dancers and choreographers—LaMoGen allows users to precisely tune attributes like energy, fluidity, and tension without the need to retrain the underlying generative model.
The Laban Movement-Guided Diffusion Framework
The core of LaMoGen is a two-step guided generation process that integrates LMA features into the diffusion pipeline. LMA categorizes movement into components such as Effort (Weight, Time, Flow) and Shape.
Methodology
- Baseline Generation: The model first generates a "neutral" motion based solely on the text prompt (e.g., "a person is stretching arms").
- Laban-Guided Optimization: The system extracts LMA features from this baseline. The user provides a "Laban tag" (e.g., "Small flow" or "Large shape"), which scales the baseline features to create a target motion profile.
- Inference-Time Guidance: During a second generation pass, the model iteratively optimizes the text embedding at each denoising step to minimize a "Laban loss." This loss measures the discrepancy between the current motion's features and the target features.
To enable backpropagation, the researchers developed differentiable LMA feature extraction formulas. They reformulated standard LMA mathematics—which often use non-differentiable "max" operators—into time-series calculations using Gaussian smoothing to handle noise in acceleration and jerk data.
Key Technical Insights
- Content-Aware Scaling: A critical realization in LaMoGen is that absolute targets for movement attributes are often meaningless because "heavy" looks different for a walk than it does for a punch. By using a two-step process, LaMoGen sets targets relative to a baseline, ensuring the style remains semantically appropriate for the action.
- Disentangled Control: The method achieves high "diagonality," meaning specific attributes like "Weight" (strength) can be adjusted independently without unintentionally affecting other qualities like "Time" (speed) or "Flow" (smoothness).
- Zero-Shot Versatility: Because optimization occurs at the embedding level during inference, LaMoGen acts as a plug-and-play module for existing pretrained diffusion models, requiring no new training data.
Experimental Results
Evaluated on the HumanML3D dataset, LaMoGen significantly outperformed traditional baselines such as simple prompt editing and standard classifier guidance.
Quantitatively, LaMoGen achieved a diagonality score of 0.978, far exceeding the 0.445 achieved by prompt editing. While there is a minor trade-off in text alignment (R-Precision) when enforcing extreme styles, the Frechet Inception Distance (FID) remains competitive, indicating that the generated motions remain realistic and high-quality.
Conclusion and Future Directions
LaMoGen provides a powerful tool for fine-grained control in motion synthesis, though it currently faces limitations regarding inference speed due to the optimization loop. Future work may involve "distilling" this inference-time control back into the model architecture for single-pass generation or conducting user studies to create more perceptually grounded mappings for Laban tags. Despite these challenges, LaMoGen represents a significant step toward more expressive and controllable AI-driven animation.
Summary updated