TL;DR
This paper introduces a novel framework that uses Riemannian geometry to create a motion space (MS) that is invariant to viewing distance and angle. This allows for more robust human motion prediction using a simple Multi-Layer Perceptron (MLP) network, overcoming limitations of previous methods that assumed consistent viewing conditions.
What Problem Does This Paper Address?
Human motion prediction is crucial for applications like autonomous driving and human behavior understanding. However, existing data-driven methods often assume that training and testing data share the same viewing distance and angle. In real-world scenarios, these viewing conditions can vary significantly, leading to inaccurate predictions. This paper tackles the challenge of achieving view-distance and view-angle invariance in human motion prediction, aiming to make models more applicable to diverse practical situations.
Methods
The core of the proposed method lies in constructing a "Motion Space" (MS) using Riemannian geometry. This MS is designed to be invariant to viewing distance and angle.
- Posture Space (PS): First, skeletal templates are normalized to eliminate view-distance variations. This creates a PS where motion is represented by the orientation of bones.
- Pre-Motion Space (pre-MS): Motions are then treated as trajectories within the PS. A novel "motion evolution distance" is introduced to measure differences based on changes in speed and direction, rather than just joint positions. This pre-MS still contains equivalence classes for view-angle variations.
- Specified Path Transport Square-Root Velocity Function (SP-TSRVF): To achieve view-angle invariance, the SP-TSRVF is introduced. This function aligns tangent spaces of motions along a specified path (rather than just a geodesic), effectively removing the influence of view-angle transformations. This transforms the pre-MS into a flattened "Motion Space" (MS).
- Sequence Predictor: A simple MLP network is then used to predict future motion sequences within this invariant MS. This approach avoids complex manifold optimization techniques.
Datasets
- Human 3.6M: A large-scale dataset used for training and evaluating the model on standard and synthetically altered (view-distance and view-angle variations) test data.
- CMU MoCap: A smaller dataset with more "violent" motions, used to test the model's performance on less ideal data.
Key Insights and Ideas
- Rethinking Motion Representation: The paper argues that representing motion solely by joint positions or rotations is insufficient when viewing conditions change. By moving to a "Motion Space" derived from Riemannian geometry, they can decouple motion content from viewing perspective.
- Motion Evolution vs. Joint Position: The proposed "motion evolution distance" is more effective than traditional joint position-based metrics for capturing the essence of motion, especially for diverse postures like walking.
- Geometric Encoding for Simplicity: Using geometric transformations (SP-TSRVF) to create an invariant space allows for the use of simple MLP networks, reducing computational complexity and parameter count compared to complex architectures.
- Invariance as a Design Principle: The framework is explicitly designed to handle view-distance and view-angle variations, making it more robust and generalizable than models trained under fixed viewing conditions.
Key Results
- State-of-the-Art Performance: The proposed method achieved competitive, and in some cases state-of-the-art, performance on both Human 3.6M and CMU MoCap datasets, particularly for short-term predictions.
- Robustness to Variations: On synthetic datasets with view-distance and view-angle variations, the proposed method showed significantly less performance degradation compared to baseline methods. For example, on a synthetic Human 3.6M dataset, it achieved a 7.2% and 10.4% reduction in AMPJPE compared to PGBIG and LTD, respectively.
- Efficiency: The MLP-based network, enabled by the invariant MS, has a low parameter count (< 0.1M), leading to faster training and inference times compared to many complex models.
- Visualization: Visualizations (Figs. 10, 11, 12) clearly demonstrate the method's ability to predict motions accurately despite input variations and show a more organized distribution of encoded motions.
Limitations and Future Work
- Performance Trade-off: The paper notes that the proposed loss function, which prioritizes robustness to postural variations, might lead to slightly less accurate long-term predictions compared to methods solely focused on joint positions. However, the outputs remain "reasonable."
- Computational Cost: While the MLP is simple, the encoding and decoding stages involving SP-TSRVF add computational overhead. The paper states the overall complexity of encoding/decoding is approximately O((N+T)²), which is manageable.
- Future Applications: The authors suggest that the proposed geometric encoding method holds potential for applications beyond motion prediction, including motion recognition, motion segmentation, and other related fields.
Summary updated