Abstract
Text-guided 3D motion editing has seen success in single-person scenarios, but its extension to multi-person settings is less explored due to limited paired data and the complexity of inter-person interactions. We introduce the task of multi-person 3D motion editing, where a target motion is generated from a source and a text instruction. To support this, we propose InterEdit3D, a new dataset with manual two-person motion change annotations, and a Text-guided Multi-human Motion Editing (TMME) benchmark. We present InterEdit, a synchronized classifier-free conditional diffusion model for TMME. It introduces Semantic-Aware Plan Token Alignment with learnable tokens to capture high-level interaction cues and an Interaction-Aware Frequency Token Alignment strategy using DCT and energy pooling to model periodic motion dynamics. Experiments show that InterEdit improves text-to-motion consistency and edit fidelity, achieving state-of-the-art TMME performance.
Introduction
While AI can now generate or edit a single person’s 3D motion with impressive fidelity, multi-person interactions remain a significant challenge. Editing an interaction is inherently more complex than editing a solo performance due to spatio-temporal coupling. For instance, changing a sequential action into a simultaneous one requires precise coordination; a slight timing delay or spatial drift can result in a chaotic collision rather than a synchronized interaction. Furthermore, a substantial data gap exists: while single-person motion data is abundant, there are few "source motion + edit instruction = target motion" triplets for multi-human scenarios. This work addresses both the data scarcity and the technical challenges of maintaining synchronization during the editing process.
Methodology
InterEdit is a conditional diffusion model designed to handle the nuances of multi-human interaction. It incorporates several key innovations:
- Symmetric Interleaved Token Aggregation: Instead of simple concatenation, the model interleaves motion tokens from both individuals. This allows the transformer backbone to effectively model the "give and take" and role-switching inherent in human interaction.
- Semantic-Aware Plan Token Alignment: The model utilizes learnable "plan tokens" aligned with high-level semantic embeddings of the target motion. These tokens act as a "director’s script," providing global guidance on the required semantic changes.
- Interaction-Aware Frequency Token Alignment: To manage timing and rhythm, the model analyzes motion in the frequency domain. By applying a Discrete Cosine Transform (DCT) to the average and difference of the two individuals' movements, it captures the rhythmic "beat" of the interaction (e.g., synchronous vs. alternating movements).
- Synchronized Classifier-Free Guidance (SCFG): During inference, the model balances source motion preservation with text prompt adherence. By dropping both conditions simultaneously during training, the model avoids bias toward either condition, resulting in cleaner edits.
Dataset: InterEdit3D
The authors introduce InterEdit3D, the first large-scale dataset specifically for multi-human motion editing. Built upon the InterHuman dataset, it was constructed using a semi-automatic pipeline where human annotators provided instructions to transform one motion into another. The dataset contains 5,161 source–target–text triplets. Alongside the dataset, the Text-guided Multi-human Motion Editing (TMME) benchmark is established to evaluate future models on instruction adherence and source preservation.
Experimental Results
InterEdit sets a new state-of-the-art for the TMME task, significantly outperforming adapted single-person editors and multi-person generators.
Quantitatively, InterEdit improved Generated-to-Target (g2t) retrieval—a measure of instruction adherence—by nearly 6% and reduced the FID—a measure of motion realism—by 16.7% compared to the strongest baseline, TIMotion. Qualitative human preference studies showed that InterEdit was preferred in over 75% of cases, particularly excelling in "Interaction Realism" due to its frequency-domain alignment strategy.
Conclusion
InterEdit represents a significant step forward in multi-human motion editing. While challenges remain—such as handling subtle gesture ambiguities and long-horizon spatial drift—the dual-alignment strategy of plan and frequency tokens provides a robust framework for capturing both the "what" and the "how" of human interactions. Future work will explore explicit geometric constraints and scaling the framework to handle larger groups of people.
The dataset and code are available at https://github.com/YNG916/InterEdit.
Summary updated