Reducing Footskate in Human Motion Reconstruction with Ground Contact Constraints

Yuliang Zou, Jimei Yang, Duygu Ceylan, Jianming Zhang, Federico Perazzi, Jia-Bin HuangWACV 2020 · DOI

Introduction

Reconstructing 3D human motion from monocular RGB video is a fundamental challenge in computer vision with applications ranging from AR/VR to digital animation. While frame-by-frame pose estimation has seen significant progress, temporal inconsistencies often lead to "jitter" and a phenomenon known as footskate. Footskate occurs when a character's feet appear to slide or drift across the floor during a stance phase, breaking physical realism and immersion. This paper introduces a robust method to eliminate footskate by combining a specialized neural network for ground contact detection with a two-stage optimization process that enforces zero-velocity constraints.

Methodology

The proposed system utilizes a three-stage pipeline to generate physically plausible motion trajectories from raw video input.

  1. Initial Pose Estimation: A state-of-the-art model (HMR) estimates the 3D pose and shape of the subject in every frame.
  2. Ground Contact Detection: The core innovation is a Temporal Convolutional Network (TCN) that predicts ground contact. Instead of raw pixels, the TCN processes 2D keypoint trajectories (extracted via OpenPose) over a 7-frame window. It outputs a 4-bit binary vector indicating whether the left/right toe and heel are in contact with the ground.
  3. Two-Stage Optimization: To avoid mathematical conflicts between smoothness and physical constraints, the authors perform optimization sequentially. First, they optimize for temporal smoothness. Second, they introduce a Zero Velocity Constraint ($L_{zv}$). If the detector identifies a foot in contact with the ground, the optimization forces that foot's global position to remain stationary between frames.
The proposed pipeline: from video input to 2D/3D pose estimation, ground contact prediction, and final motion reconstruction.

Data and Implementation

To train the contact detector, the authors created a new semi-automatic dataset of ~75,000 frames. They used motion capture data from Human3.6M and MADS to calculate foot velocities, labeling frames as "in contact" when velocity fell below a specific threshold, followed by manual refinement.

A key insight from their implementation is that 2D keypoints alone provide sufficient motion cues for contact detection, making the model robust to varying environments without needing complex optical flow. They also emphasize the use of a OneEuro filter to pre-process noisy 2D keypoints, ensuring the TCN receives clean input data.

Results

The system demonstrates significant improvements in both quantitative accuracy and qualitative realism. The contact detector achieved a Mean Average Precision (mAP) of 0.9292, outperforming standard velocity-thresholding heuristics. On the Human3.6M benchmark, the method reduced the Mean Per-Joint Position Error (MPJPE) from 71.05mm to 69.63mm.

Qualitative comparison showing how the proposed method (Ours) correctly plants the foot on the ground compared to standard frame-by-frame (HMR) and smoothed (SFV) baselines.

Visually, the results are striking. As shown in the comparison above, the "sliding" effect prevalent in previous methods is almost entirely removed. The reconstructed humans appear firmly rooted to the ground, even during complex movements like dancing or sports.

Discussion and Future Work

While the method effectively eliminates footskate on flat surfaces, it currently assumes a horizontal ground plane. Future work could extend this to uneven terrain, stairs, or slopes. Additionally, the current model focuses exclusively on foot-to-ground contact; expanding the framework to include hand-object interactions or sitting poses would further enhance the versatility of monocular motion reconstruction. Despite these limitations, the explicit imposition of physical constraints represents a major step toward generating production-quality human motion from simple video sources.

Summary updated

BibTeX
@inproceedings{zou2020reducing,
  author = {Yuliang Zou and Jimei Yang and Duygu Ceylan and Jianming Zhang and Federico Perazzi and Jia-Bin Huang},
  title = {{Reducing Footskate in Human Motion Reconstruction with Ground Contact Constraints}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  doi = {10.1109/wacv45572.2020.9093329},
}
Figure