Text-to-motion models turn a sentence such as "crouch, then jump, then wave with the left hand" into a 3D animation of a human skeleton. They are good at producing motions that look human. They are bad at the logical content of the sentence. They drop the middle action, swap the order, or wave the wrong hand, and these failures get worse as prompts move away from the short, everyday captions they were trained on. Recent work tunes these models with preference learning: the model is shown pairs of its own outputs ranked "better" and "worse" and pushed toward the better one. The ranking comes from a single similarity score produced by a text-motion retrieval model. That score is known to be largely blind to order and omission, so the preference signal carries little information about exactly the errors that matter.
We propose to supply that signal by construction instead of by judgment. From densely annotated motion capture, we build minimal pairs: two real, physically plausible motions that differ in one logical property, such as the order of two actions, which side of the body performs an action, or whether an intermediate action happens at all. Each pair is matched with a programmatically generated description whose meaning is known exactly. Because we know which motion is correct for which sentence, the pair needs no learned judge. This is the motion analogue of the verifiable answer checkers that make preference training work for math reasoning. We then fine-tune a diffusion motion generator (a model that learns to turn random noise into samples by gradual denoising) with a preference objective on these pairs, anchored by a likelihood term on the correct motions. The template-generated descriptions are paraphrased and evaluated on human-written captions, so that the model learns the logic rather than the template wording.
Core Claim
Preference training for text-to-motion fails on compositional prompts mainly because its negatives are uninformative, not because preference training is the wrong tool. Negatives that are real motions, logically wrong for the prompt but otherwise indistinguishable from correct ones, should teach order, handedness and completeness. Retrieval-scored self-generated negatives do not. The prediction is specific. On the multi-event subset of HumanML3D-E, and on programmatic checks of order and handedness, minimal-pair preference training should beat retrieval-judged preference training (MoDiPO, SoPo). It should also beat plain supervised training on the same edited motions with corrected captions, while not hurting standard realism metrics.
Where This Sits
Two motion papers apply preference optimization to text-to-motion diffusion, and both motivate this work by exposing its limits. MoDiPO ranks candidates generated by Motion Latent Diffusion (MLD) using retrieval similarity and applies DPO, the direct preference optimization objective. SoPo keeps human motion capture as the preferred sample and uses the model's current outputs as the dispreferred one. In both, the negative is "whatever the generator happens to get wrong," scored holistically, and SoPo's authors name the scalar judge as their bottleneck. Event-T2M supplies the key constraint: retrieval encoders are insensitive to reordering and omission, and models collapse on prompts with four or more events. Event-T2M fixes this architecturally with event-level cross-attention and still trains with a plain denoising loss. Our method changes the training signal instead, so it is complementary to that architecture. TriC-Motion also uses the word "counterfactual," but it means statistical diffusion-noise confounders, not semantic alternatives. EMS and FG-MDM decompose prompts into atomic actions or body parts through fixed templates or unconstrained LLM rewriting, and neither trains against wrong outputs.
The method itself is borrowed from the reasoning literature. Iterative RPO shows that preference training on outcome-verified correct and incorrect chains, with a likelihood anchor on the winner, yields large gains: Llama-2-70B-Chat on GSM8K went from 55.6% to 81.6%. It also argues that without the anchor, DPO lowers the probability of both answers. PORT and MAMUT build hard negatives by surgical corruption of digits or formulas, and Chain of Preference Optimization localizes negatives at the exact branch where reasoning diverges. Our minimal pairs carry this to continuous motion: the corruption is a known, single logical edit. LEMA reports that training on mistake-correction pairs beat training on an equal volume of extra correct chains. That is the main empirical reason to expect negatives to add something over positives, though it was shown in text, not motion.
Template work sets the constraint on the language side. COVR shows models falling from about 68% to as low as 1.6% on held-out compositional structures even with paraphrasing, and argues that logical forms leak through templates. Contrast sets from scene graphs shows that template-based probes test only the templated structures. MUTANT trains with paired semantic mutations of the input, with a loss that makes output differences track meaning differences. It does so for categorical VQA answers, and we extend the idea to generated motion sequences. The lesson we take is that templates are valuable for their guaranteed semantics and harmful as a training distribution of surface forms. We therefore use them to define pairs, paraphrase them away during training, and judge success only on human-written captions.
Research Program
Pair construction. The source is BABEL, which gives per-frame action labels and transition boundaries over 13,220 AMASS sequences. We build three edit families. For order pairs, we mine real sequences that contain the same two or three labeled actions in different orders; where mining is too sparse, we compose segments through a single shared blending pipeline. For handedness pairs, we apply a left-right skeletal mirror, which is exact and physically valid, to segments whose labels are side-specific. For omission pairs, we contrast A–B–C with A–C, both built the same way.
Each construction route is applied identically to winners and losers. For example, when the loser is a stitched composition, the winner is too. The edit then changes only logic, not artifact statistics. This matters because the reasoning literature warns that models learn surface artifacts of negatives. As a check, a text-blind classifier must fail to tell winners from losers. Descriptions come from a small grammar over BABEL labels with operators for sequence, simultaneity and side. Each description is paraphrased several times by an LLM, and whole template families are held out.
Objective. Every pair is used in both directions, in the spirit of MUTANT. Take motions $m$ and $m'$ that differ by one edit, with descriptions $c$ and $c'$. Then $m$ is the winner under $c$ and the loser under $c'$, and the reverse holds for $m'$. Following the diffusion form of DPO on an MLD backbone, as in MoDiPO, and adding the anchor from Iterative RPO, we minimize
$$\mathcal{L} = -\log\sigma\!\Big(-\beta\big[(e_\theta(x^w)-e_{\text{ref}}(x^w)) - (e_\theta(x^l)-e_{\text{ref}}(x^l))\big]\Big) + \alpha\, e_\theta(x^w).$$
Here $e_\theta(x)=\lVert \epsilon-\epsilon_\theta(x_t,t,c)\rVert^2$ is the denoising error of the model being trained on the noised latent $x_t$ of motion $x$ at step $t$ with noise $\epsilon$ and description $c$. $e_{\text{ref}}$ is the same error under the frozen starting model. $x^w$ and $x^l$ are the winner and loser, $\beta$ sets the preference sharpness, and $\alpha$ weights the likelihood anchor on the winner. The anchor term is ordinary supervised training on the correct edited motions. This means the method strictly contains the obvious augmentation baseline, and the experiments isolate what the preference term adds.
Optional mixing. In a second stage we mix in SoPo-style semi-online pairs. Here the loser is the model's own generation, labeled wrong by the programmatic checks below rather than by retrieval similarity. This tests whether constructed pairs and self-generated errors combine.
Baselines
The naive reference is the pretrained MLD model with no further tuning. All methods start from this checkpoint. The strongest existing preference method on this backbone is SoPo, which reports a 3.25% Multi-Modal Distance improvement on MLD against 0.76% for MoDiPO. MoDiPO reports HumanML3D FID moving from 0.459 to 0.281. We rerun both from their published recipes on the same MLD checkpoint, with the same number of preference updates and the same hyperparameter search budget as our method. This keeps the comparison about the signal and not about tuning effort.
The strongest architectural competitor on the target regime is Event-T2M. It reports FID 0.265 on the HumanML3D-E subset with four or more events, against 0.418 for MoMask and 1.077 for AttT2M. We compare against its reported number and, if its code is available, also apply our fine-tuning on top of it to test complementarity. For the order and handedness metrics defined below, no reported result was found on Lacuna for any method, so all such numbers will be produced by us under identical evaluation.
Two ablations isolate the key ingredient. The first is augmentation-only: the anchor term alone, trained on exactly the same edited motions with their correct captions, with the same number of gradient steps. The second removes the anchor, $\alpha=0$, to test whether the DPO sink identified by Iterative RPO appears in motion diffusion as lost diversity. Beating the field means two things. On the four-or-more-event subset, our method must improve order accuracy and handedness accuracy over SoPo and over augmentation-only, with non-overlapping seed confidence intervals. It must also keep HumanML3D FID and diversity no worse than SoPo's.
Why not just…?
Why not just add the edited motions as extra supervised data with corrected captions? This is the most serious threat. Every loser in our pairs is a valid positive for the flipped description, so plain supervised training sees the same motions. LEMA's finding that corrections beat equal-volume extra correct data comes from text reasoning and may not transfer. There is a mechanistic reason it might transfer: a frame-wise denoising error spends most of its gradient on pose detail, while the preference term concentrates gradient on the one dimension along which the paired motions differ. That is a hypothesis, not evidence. The augmentation-only ablation at matched data and steps settles it. If it matches the full method on order and handedness, the honest conclusion is that the constructed data, not the contrastive objective, carries the gain, and we will report it that way.
Why not just train a better, order-aware judge on these pairs and plug it into SoPo? This is also plausible. The pairs could train a temporally sensitive reward model, and SoPo would then score the model's own outputs with it. Its advantage is that it sees the generator's actual error distribution. Its risk is that learned judges fail off-distribution, which SoPo's authors themselves name as their limitation. We run this variant at matched compute. The semi-online stage, which uses programmatic checks as the judge, is our version of the same idea without a learned judge. Comparing the two on held-out template families shows which generalizes.
Why not just decompose prompts with an LLM at inference time, as FG-MDM and SnapMoGen do? Rewriting cannot fix handedness when the text encoder itself confuses left and right, as MotionCLIP observes for CLIP. Rewritten long descriptions also still collapse on standard backbones, as Event-T2M shows with MoMask and AttT2M. Still, we run an LLM-expansion baseline with the same generator, using event-wise expanded prompts, to measure how much of the gap rewriting closes on its own. Our method is compatible with such rewriting at test time.
Proposed Experiments
Training pairs come from BABEL. The main test is the HumanML3D-E stratification by event count, whose captions are human-written. Good performance there therefore cannot come from template wording. We also report standard HumanML3D and KIT-ML FID, R-precision, Multi-Modal Distance and diversity, which serve as a guard against realism loss.
Logic is scored programmatically. For order accuracy, each generated motion is split at low-velocity boundaries, each segment is matched to the event phrases with a retrieval encoder, and the alignment must be monotonic. Retrieval is reliable at this short, single-action scale even if it is not at the sequence scale. Handedness accuracy compares left and right wrist and ankle displacement energy on side-specific prompts. Omission is checked by event coverage. Because BABEL action classifiers are weak (2s-AGCN reaches 44.87% top-1 on the 60-class subset per BABEL), segment matching uses only coarse, well-separated action groups. We validate the evaluator against BABEL ground-truth segments before using it.
Four checks decide the outcome. The first is the augmentation-only comparison described above. The second is a leakage test: the method is trained on raw templates versus paraphrases, with template families held out, and the order accuracy gap between templated and human-written test prompts is the measured quantity, following COVR's warning. The third is the shortcut test: a text-blind classifier on winners versus losers must be near chance, otherwise the pairs teach artifact detection. The fourth is a diversity and FID trajectory over training with and without the anchor, to see whether unanchored motion DPO quietly loses diversity, as the Iterative RPO analysis predicts.
References
- MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization
- SoPo: Text-to-Motion Generation Using Semi-Online Preference Optimization
- Event-T2M: Event-level Conditioning for Complex Text-to-Motion Synthesis
- TriC-Motion: Tri-Domain Causal Modeling Grounded Text-to-Motion Generation
- Breaking The Limits of Text-conditioned 3D Motion Synthesis with Elaborative Descriptions
- FG-MDM: Towards Zero-Shot Human Motion Generation via ChatGPT-Refined Descriptions
- Iterative Reasoning Preference Optimization
- PORT: Preference Optimization on Reasoning Traces
- MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training
- Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
- Can LLMs Learn From Mistakes? An Empirical Study on Reasoning Tasks
- COVR: A Test-Bed for Visually Grounded Compositional Generalization with Real Images
- Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA
- MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering
- BABEL: Bodies, Action and Behavior With English Labels
- MotionCLIP: Exposing Human Motion Generation to CLIP Space