Abstract
Predicting human motion is critical for autonomous systems, yet high-quality datasets are scarce. We present MotionAug, an augmentation framework that combines generative modeling with physical correction. MotionAug utilizes a modified Variational AutoEncoder (VAE) and Inverse Kinematics (IK) to synthesize candidate motions, which are then refined through a physics simulator to ensure physical plausibility. To address simulation bias and computational overhead, we introduce a PD-residual force mechanism and a debiasing step. Experimental results on the HDM05 dataset demonstrate that MotionAug significantly enhances the performance of motion prediction models, particularly in data-constrained scenarios.
Introduction
Deep learning models for human motion prediction require vast amounts of high-quality data. Traditional data augmentation techniques, such as adding noise or simple geometric transformations, often result in physically implausible motions—such as "floating" characters or limbs penetrating the floor. These artifacts degrade model performance. MotionAug addresses this by creating a pipeline that synthesizes diverse motions and corrects them using the laws of physics.
Methodology
The MotionAug framework consists of three primary stages: motion synthesis, physical correction, and debiasing.
1. Motion Synthesis
The synthesis stage employs two methods to generate diverse motion candidates:
- Modified VAE: A Variational AutoEncoder trained with an adversarial loss to maintain motion sharpness. To handle small datasets, a "sampling-near-samples" strategy is used, where new motions are sampled from latent space regions close to clusters of real training data, avoiding "unlearned" areas that produce unrealistic poses.
- Semi-Automatic IK: Users define target spaces for specific joints (e.g., a foot's landing zone), and the system automatically generates valid Inverse Kinematics-based sequences.
2. Motion Correction
Synthesized motions often contain physical artifacts like "footskating." MotionAug uses imitation learning within the Bullet Physics simulator to rectify these. A simulated character attempts to mimic the synthesized motion, and the simulator's physics engine naturally filters out impossible movements. To accelerate this computationally expensive process, a PD-residual force mechanism is introduced, reducing training time from 30 hours to approximately 9 hours by narrowing the learning search space.
3. Motion Debiasing
Physics simulators typically use rigid-body approximations that do not perfectly capture human soft-tissue dynamics. This creates a "dynamic mismatch." MotionAug employs a neural network to map simulated motions back to the distribution of real human movement, ensuring the augmented data remains relevant for real-world applications.
Experimental Results
The framework was evaluated using the HDM05 Motion Database, which features dynamic actions like punching and kicking. Testing across RNN, GCN, and Transformer architectures showed consistent improvements.
As shown in the qualitative comparison, the proposed VAE avoids the "mode collapse" seen in GANs and the over-smoothed, static poses typical of vanilla VAEs when data is scarce. Quantitatively, MotionAug reduced prediction error significantly; for instance, in RNN-based "kick" motion prediction, error at 400ms dropped from 2.45 to 1.80 radians.
Conclusion
MotionAug effectively bridges the gap between generative data augmentation and physical realism. By utilizing physics as a regularizer and introducing efficient correction mechanisms, the framework provides a robust solution for training motion prediction models on limited datasets. Future work will focus on scaling the correction phase via GPU acceleration and extending the method to 2D video inputs.
Summary updated