Unsupervised and Generic Short-Term Anticipation of Human Body Motions

Kristina Enes, Hassan Errami, Moritz Wolter, Tim Krake, Bernhard Eberhardt, Andreas Weber, Jörg Zimmermann

Introduction

Predicting human body movement in the immediate future is critical for applications ranging from safe human-robot interaction to high-fidelity computer animation. While deep learning models, particularly Recurrent Neural Networks (RNNs), currently define the state-of-the-art, they often function as "black boxes" requiring extensive training on massive datasets. This research explores an alternative: Dynamic Mode Decomposition with delays (DMDd), a classical tool from dynamical systems theory that offers an unsupervised, "equation-free" approach to motion anticipation.

Methodology: The Power of Linear Dynamics

The researchers propose using DMD, a technique originally used in fluid mechanics to decompose complex time-series data into spatial patterns (modes) that oscillate and evolve over time. To adapt this to human motion, they employ Takens’ Theorem to create "delay coordinates." By stacking previous frames of motion, the model captures latent physical information like velocity and acceleration that is not visible in a single snapshot.

Unlike RNNs, DMDd is unsupervised and generic. It does not require a training phase on specific actions like "walking" or "sitting." Instead, it analyzes the physics of the current motion sequence in real-time to predict the next few hundred milliseconds, making it faster and more interpretable than neural network counterparts.

Experimental Evaluation

The approach was tested using the Human3.6M dataset, focusing on Actor #5 across various activities. The researchers compared DMDd against a baseline RNN for anticipation horizons between 0.1 and 0.4 seconds.

Comparison of MSE for RNN and DMDd across different anticipation times.

Key Results

The experiments revealed that for very short-term anticipation (0.1s to 0.2s), DMDd significantly outperforms RNNs. As shown in the figure above, the Mean Squared Error (MSE) for DMDd remains remarkably low at 0.1s and 0.2s compared to the RNN baseline. However, as the anticipation time reaches 0.4s, the error for DMDd increases sharply, suggesting a limit to the linear assumption as human intent begins to play a larger role than physical momentum.

The performance gains are particularly evident when looking at specific action classes. For instance, in the "Directions" task at a 0.1s horizon, the RNN's MSE was 787.18, while DMDd achieved a mere 9.09.

Detailed MSE comparison across various action classes for RNN and DMDd at different time horizons.

Insights and Limitations

  • The "Sweet Spot" for Memory: The model's accuracy depends on the number of delays used. The researchers found that using 40 to 90 delays typically minimizes error, balancing the capture of dynamics against noise overfitting.
  • The 0.4s Threshold: The effectiveness of DMDd wanes after 0.4 seconds. At this point, human intent (e.g., deciding to turn or stop) begins to override the physical momentum captured by the linear model.
  • Computational Efficiency: Because DMDd relies on Singular Value Decomposition (SVD) and linear regression rather than iterative backpropagation, it is significantly faster and more computationally efficient than neural network training.

Conclusion

This research demonstrates that for the immediate "physical" horizon of human motion, simple linear models can outperform complex neural networks. The authors suggest that future systems might benefit from a hybrid architecture: using DMDd for ultra-fast, short-term physical anticipation and deep learning for longer-term, intent-based prediction.

Summary updated

BibTeX
@misc{enes2019unsupervised,
  author = {Kristina Enes and Hassan Errami and Moritz Wolter and Tim Krake and Bernhard Eberhardt and Andreas Weber and Jörg Zimmermann},
  title = {{Unsupervised and Generic Short-Term Anticipation of Human Body Motions}},
  year = {2019},
  eprint = {1912.06688},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1912.06688},
}
Figure