Abstract
This paper introduces the Conditional Spatial-Temporal Variational Graph Auto-Encoder (CST-VGAE), a novel framework for long-term head pose forecasting. Unlike current methods that often produce unrealistic or over-smoothed results due to the "one-to-many" nature of future movements, CST-VGAE leverages human physiological priors—specifically, the tendency for gaze to shift before the head moves. By integrating a gaze-guiding prior with a structured Gaussian Mixture Model (GMM) latent space and temporal self-attention, the model achieves state-of-the-art performance on benchmarks like the BIWI and UPNA datasets, providing more accurate and realistic 3D head orientation predictions over long time horizons.
Introduction
Head pose estimation is a well-established field, but forecasting future poses remains a significant challenge. The primary difficulty lies in the "one-to-many" mapping: for any given current state, there are multiple plausible future trajectories. Deterministic regression models often fail by predicting a single "average" future, resulting in blurry or unrealistic outputs. While generative models like standard VAEs offer more diversity, they frequently lack the necessary constraints to maintain realism in long-term scenarios. This work proposes to constrain the forecasting space using physiological priors and structural graph representations.
Methods
The proposed CST-VGAE framework utilizes several key innovations to improve forecasting accuracy and realism.
Gaze-Guiding Prior
A central contribution is the use of gaze direction as a conditional input. Physiological research indicates that gaze shifts at time $t$ are strong predictors of head pose at time $t+l$. The model exploits this correlation to "steer" the generative process toward plausible futures. Empirical analysis shows that the Pearson Correlation Coefficient (PCC) between gaze and head pose peaks at specific frame intervals, justifying this predictive relationship.
CST-VGAE Architecture
The architecture employs a Gaussian Mixture Model (GMM) latent space with three components corresponding to the three degrees of freedom: yaw, pitch, and roll. This structured approach allows for a more interpretable and effective latent representation than a single Gaussian. A temporal self-attention module captures long-range dependencies between past gaze and predicted movements, while self-supervised learning on future landmarks further refines the output.
Results
The model was evaluated on the BIWI Kinect Head Pose Database and the UPNA Head Pose Database for generalizability. On the BIWI dataset, CST-VGAE achieved a Mean Absolute Error (MAE) of 6.27 for 30-step-ahead forecasts, significantly outperforming baselines like GAZE-LSTM (9.12) and standard VGAE (9.21). Cross-dataset evaluations on UPNA confirmed the model's robustness. Ablation studies demonstrated that both the gaze-guiding prior and the GMM latent space are critical to the model's superior performance.
Conclusion
CST-VGAE demonstrates the power of incorporating physiological constraints into generative models for human motion forecasting. While the model shows some limitations in static scenarios and depends on the quality of upstream landmark detectors, its ability to produce realistic, long-term head pose predictions opens new possibilities for applications in driver assistance systems and immersive VR/AR environments.
Summary updated