Abstract
Most human pose forecasting models struggle with "jittery" or physically impossible predictions because they only apply constraints during training, not during actual use. This paper introduces Temporal Constrained Subspace Learning (TCSL), a method that explicitly maps predictions into a "feasible subspace" during inference. By integrating a differentiable projection layer into the model architecture, TCSL guarantees that predicted movements stay within realistic physical bounds without requiring expensive iterative calculations at test time.
Introduction
Human pose forecasting—predicting future body movements based on historical skeletal data—is a critical component for autonomous driving, human-robot collaboration, and healthcare monitoring. While deep learning models have achieved significant success in minimizing average joint position errors (MPJPE), they frequently fail basic physical "sanity checks."
Standard models often treat physical constraints, such as temporal continuity, as "soft" suggestions via loss functions during training. Consequently, during the inference stage, these constraints are no longer enforced. If the input data deviates from the training distribution, the model may predict joints moving at speeds or accelerations that are biologically impossible. This paper addresses the gap between statistical accuracy and physical feasibility.
Methodology: Temporal Constrained Subspace Learning
The authors propose a framework that forces model outputs to reside within a Temporal Constrained Subspace. Unlike previous methods that rely on slow, iterative optimization to "fix" poses after prediction, TCSL is a direct, feed-forward operation embedded in the network architecture.
The process consists of four primary steps:
- Subspace Transformation ($\mathcal{H}$): Instead of directly predicting raw 3D coordinates, the model's output is transformed into a subspace representing the second-order derivative of the pose (change of velocity).
- Dynamic Constraints: The system analyzes historical motion to establish a "feasibility bound" ($\delta_{ub}$), assuming that future changes in velocity should not drastically exceed observed past behavior.
- The Projection Operation ($\mathcal{P}$): This core innovation uses a differentiable projection function (a modified ReLU) to "clamp" any predicted values falling outside the feasible bound back into the allowed range.
- Inverse Transformation ($\mathcal{H}^{-1}$): Finally, the constrained values are mapped back into standard 3D coordinates for the final output.
Experimental Evaluation
The researchers evaluated TCSL on three major benchmarks: Human3.6M (controlled environment), AMASS (unified motion capture), and 3DPW (real-world outdoor sequences).
The study introduced a new metric, the Infeasible Rate (IR), to quantify how often a model violates temporal constraints. While state-of-the-art models like STS-GCN exhibited an IR of over 3% in long-term forecasts, the TCSL-enhanced model maintained a 0% IR by design.
Key Results and Discussion
The results demonstrate that physical realism serves as a powerful inductive bias. Surprisingly, forcing the model into a feasible subspace did not just improve physical consistency; it also reduced the overall prediction error (MPJPE). On Human3.6M, TCSL outperformed existing methods across nearly all action categories, such as reducing the "Walking" error at 1000ms from 51.8mm to 50.7mm.
As shown in the distribution analysis, baseline models often produce "tails" of high-magnitude velocity changes that do not exist in the ground truth. The TCSL projection effectively eliminates these outliers, resulting in qualitatively smoother and more natural motion.
Conclusion and Future Work
TCSL represents a shift from "soft" constraints in loss functions to "hard" constraints within the model architecture. While the current implementation focuses on temporal feasibility, the authors acknowledge that spatial constraints—such as bone length consistency and joint angle limits—are not yet addressed. Future work aims to integrate these spatial dimensions and explore the application of feasible subspace learning to other sequential tasks, including trajectory prediction for autonomous vehicles and path planning for robotics.
Summary updated