Recovering Human Motion in Global Coordinates

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2172 papers199 conceptsfrozen 2026-10-04

Most 3D human mesh recovery (HMR) systems suffer from a perspective trap: they see the world through the lens of a camera-centric coordinate system. In this view, if a person walks forward while the camera backs away at the same speed, the person appears stationary. This coupling of ego-motion and subject-motion makes it impossible to know where a person actually is in the world.

Recent research has shifted toward World-Space Recovery, a paradigm that decouples the movement of the observer from the movement of the subject. By placing humans into a stable, global coordinate frame, these methods allow us to track trajectories across large environments, even when the camera is handheld, mounted on a drone, or moving through a crowd.

The Camera Centric Limitation

Traditional models like VIBE by Muhammed Kocabas and Nikos Athanasiou excel at capturing local joint rotations—the "pose"—but often fix the root joint (the pelvis) at the origin. As noted by Ye Yuan 0007 and Pavlo Molchanov in their work on GLAMR, this causes the reconstructed global trajectory to become distorted or lost entirely when the camera moves.

Without world-space context, we see physical artifacts like "foot skating" (where feet slide across the floor) or "moonwalking" (where a character performs a walking animation but remains spatially static).

The Human as a Biometric Ruler

The fundamental challenge in monocular world-space recovery is scale ambiguity. From a single image, it is difficult to tell if a person is small and close or large and far away. To solve this, researchers like Wanqi Yin and Zhongang Cai in WHAC propose using the human body itself as a "biometric ruler."

Because human proportions (like bone lengths) are relatively constant, the system can use the detected person to calibrate the camera's movement. If the person's 2D projection shrinks while their 3D pose remains a "stride," the system can infer the absolute distance the person moved relative to the camera. Tuanfeng Yang Wang and Yizhou Zhao extend this in SynCHMR by using these body dimensions to calibrate Simultaneous Localization and Mapping (SLAM), ensuring the camera trajectory and human motion are physically consistent.

Decoupling and Joint Optimization

To achieve a stable world-space reconstruction, the system must solve for three things simultaneously: the camera's path, the person's local pose, and the person's global trajectory.

Vickie Ye introduced SLAHMR, which uses learned human motion priors to "anchor" the scene. By enforcing that humans must touch the ground and move at plausible velocities, the model can reconstruct multiple people in a shared 3D world frame even in "in-the-wild" videos where scene geometry is unknown. Similarly, TRAM by Yufu Wang and Kostas Daniilidis robustifies this process by masking out the moving humans during SLAM execution, preventing the person's movement from confusing the camera's ego-motion estimation.

Hallucinating Through Occlusions

In real-world videos, people often walk behind trees, cars, or completely out of the frame. Standard HMR fails the moment the person is obscured. GLAMR addresses this by combining a global trajectory predictor with a generative motion "infiller."

By learning the relationship between local body language (e.g., the lean of a torso) and global movement (e.g., turning a corner), the model can "hallucinate" plausible trajectories. Even if a person is hidden for several seconds, the system can predict where they will re-emerge in the global coordinate system based on their momentum and previous pose sequence.

Lifting 2D to the 3D World

A newer frontier involves using generative diffusion models to bridge the gap between 2D observations and 3D world-space motion. Jiaman Li and Jiajun Wu 0001 developed MVLift, which reconstructs global trajectories from single-view 2D poses without any 3D ground truth.

MVLift works by arranging virtual cameras in a circular array around the 2D sequence and enforcing multi-view consistency. By optimizing for a motion that looks correct from every virtual perspective, the model naturally "lifts" the 2D joints into a 3D trajectory that includes root translation—something local-only models cannot achieve.

Why World Space Matters

The shift from camera-relative to world-grounded recovery is not just a technical refinement; it is a requirement for downstream applications:

  • Robotics: Dantong Niu and Yuvan Sharma show in ARM4R that 4D representations (3D tracks over time) are essential for robots to learn physical geometry from human videos.
  • Interaction Modeling: Models like InterGen by Han Liang demonstrate that multi-person interactions (like dancing or fighting) require a shared global coordinate system to prevent characters from drifting apart or overlapping.
  • Real-time Capture: Systems like ProxyCap by Hongwen Zhang now achieve this world-space tracking at 30 FPS on standard laptops, bringing professional-grade motion capture to everyday mobile devices.

Papers in this version

Showing 60 of 172 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 90 papers
  2. Version 1 · 2026-10-04 · 172 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_absolute_spatial_localization_9ca5e48d,
  title = {absolute spatial localization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/absolute-spatial-localization/rdr_9ca5e48d57764681acc00968bd372414},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_recovering_human_motion_in_global_coordi_9ca5e48d_v1,
  title = {Recovering Human Motion in Global Coordinates},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/recovering-human-motion-in-global-coordinates/rdr_9ca5e48d57764681acc00968bd372414/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}