Temporal Constrained Feasible Subspace Learning for Human Pose Forecasting

Gaoang Wang, Mingli Song

Abstract

Most human pose forecasting models struggle with "jittery" or physically impossible predictions because they only apply constraints during training, not during actual use. This paper introduces Temporal Constrained Subspace Learning (TCSL), a method that explicitly maps predictions into a "feasible subspace" during inference. By integrating a differentiable projection layer into the model architecture, TCSL guarantees that predicted movements stay within realistic physical bounds without requiring expensive iterative calculations at test time.

Introduction

Human pose forecasting—predicting future body movements based on historical skeletal data—is a critical component for autonomous driving, human-robot collaboration, and healthcare monitoring. While deep learning models have achieved significant success in minimizing average joint position errors (MPJPE), they frequently fail basic physical "sanity checks."

Standard models often treat physical constraints, such as temporal continuity, as "soft" suggestions via loss functions during training. Consequently, during the inference stage, these constraints are no longer enforced. If the input data deviates from the training distribution, the model may predict joints moving at speeds or accelerations that are biologically impossible. This paper addresses the gap between statistical accuracy and physical feasibility.

Methodology: Temporal Constrained Subspace Learning

The authors propose a framework that forces model outputs to reside within a Temporal Constrained Subspace. Unlike previous methods that rely on slow, iterative optimization to "fix" poses after prediction, TCSL is a direct, feed-forward operation embedded in the network architecture.

Overview of the TCSL framework, illustrating the transformation into a constrained subspace and the projection operation.

The process consists of four primary steps:

  1. Subspace Transformation ($\mathcal{H}$): Instead of directly predicting raw 3D coordinates, the model's output is transformed into a subspace representing the second-order derivative of the pose (change of velocity).
  2. Dynamic Constraints: The system analyzes historical motion to establish a "feasibility bound" ($\delta_{ub}$), assuming that future changes in velocity should not drastically exceed observed past behavior.
  3. The Projection Operation ($\mathcal{P}$): This core innovation uses a differentiable projection function (a modified ReLU) to "clamp" any predicted values falling outside the feasible bound back into the allowed range.
  4. Inverse Transformation ($\mathcal{H}^{-1}$): Finally, the constrained values are mapped back into standard 3D coordinates for the final output.

Experimental Evaluation

The researchers evaluated TCSL on three major benchmarks: Human3.6M (controlled environment), AMASS (unified motion capture), and 3DPW (real-world outdoor sequences).

The study introduced a new metric, the Infeasible Rate (IR), to quantify how often a model violates temporal constraints. While state-of-the-art models like STS-GCN exhibited an IR of over 3% in long-term forecasts, the TCSL-enhanced model maintained a 0% IR by design.

Key Results and Discussion

The results demonstrate that physical realism serves as a powerful inductive bias. Surprisingly, forcing the model into a feasible subspace did not just improve physical consistency; it also reduced the overall prediction error (MPJPE). On Human3.6M, TCSL outperformed existing methods across nearly all action categories, such as reducing the "Walking" error at 1000ms from 51.8mm to 50.7mm.

Comparison of velocity change distributions between baseline and constrained predictions against ground truth.

As shown in the distribution analysis, baseline models often produce "tails" of high-magnitude velocity changes that do not exist in the ground truth. The TCSL projection effectively eliminates these outliers, resulting in qualitatively smoother and more natural motion.

Conclusion and Future Work

TCSL represents a shift from "soft" constraints in loss functions to "hard" constraints within the model architecture. While the current implementation focuses on temporal feasibility, the authors acknowledge that spatial constraints—such as bone length consistency and joint angle limits—are not yet addressed. Future work aims to integrate these spatial dimensions and explore the application of feasible subspace learning to other sequential tasks, including trajectory prediction for autonomous vehicles and path planning for robotics.

Summary updated

Abstract

Human pose forecasting is a sequential modeling task that aims to predict future poses from historical motions. Most existing approaches focus on the spatial-temporal neural network model design for learning movement patterns to reduce prediction errors. However, they usually do not strictly follow the temporal constraints in the inference stage. Even though a small Mean Per Joint Position Error (MPJPE) is achieved, some of the predicted poses are not temporal feasible solutions, which disobeys the continuity of the body movement. In this paper, we consider the temporal constrained feasible solutions for human pose forecasting, where the predicted poses of input historical poses are guaranteed to obey the temporal constraints strictly in the inference stage. Rather than direct supervision of the prediction in the original pose space, a temporal constrained subspace is explicitly learned and then followed by an inverse transformation to obtain the final predictions. We evaluate the proposed method on large-scale benchmarks, including Human3.6M, AMASS, and 3DPW. State-of-the-art performance has been achieved with the temporal constrained feasible solutions.
BibTeX
@inproceedings{wang2023temporal,
  author = {Gaoang Wang and Mingli Song},
  title = {{Temporal Constrained Feasible Subspace Learning for Human Pose Forecasting}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/161},
}
Figure