When modeling human or object movement, a fundamental challenge arises from how we represent space. If a neural network attempts to track or generate a sequence using absolute coordinates, it must simultaneously learn two distinct pieces of information: where an agent is going in the environment, and how the agent's parts are moving relative to themselves.
Conflating these two dynamics creates a massive, highly redundant search space. A model tracking a person walking across a room must learn what "walking" looks like at coordinate $(0,0)$, coordinate $(10,10)$, and every point in between. This spatial conflation severely limits the efficiency, scalability, and accuracy of computer vision models.
To solve this, a prominent research direction focuses on global-local motion disentanglement. By explicitly factorizing motion into a global trajectory (the macro-movement of a root center through space) and local dynamics (the micro-movements of limbs or deformations relative to that center), models can learn stable, translation-invariant representations of physical actions.
The Problem of Spatial Conflation
In early latent variable models for human motion, integrating local joint configurations with global spatial translation into a single unified representation proved exceptionally difficult. As noted by Raquel Urtasun et al., this difficulty often forced researchers to rely on separate, oversimplified tracking models for spatial positioning, leading to severe inconsistencies between a body's internal pose and its external trajectory.
When a model attempts to process raw video or absolute coordinate streams without this separation, it faces a high risk of confusing an agent's position on the screen with its specific physical actions. Aram Davtyan et al. demonstrate that failing to isolate global 2D spatial shifts from local object deformations forces a system to model every possible movement at every possible coordinate. This lack of structural bias leads to visual artifacts, high computational waste, and poor perceptual quality in synthesized videos.
The Mechanics of Disentanglement
The core intuition behind global-local disentanglement is to establish a moving, body-centric coordinate system. Instead of tracking joints in absolute space, the motion is factorized into two layers:
- Global Component: The trajectory of a "root joint" (typically the pelvis or center of mass), a bounding box's rigid transformation, or the overall "world flow" relative to the environment.
- Local Component: The kinematic tree of the skeleton with the root fixed, local time-varying deformation fields, or "subject-relative flow."
Absolute Motion
├── Global Trajectory (Root Translation / World Flow) ──► Macro Navigation
└── Local Dynamics (Root-Relative Joint Angles) ──► Micro Expression
As Ehsan Adeli et al. show, modeling the movement of a root joint independently from relative joint positions isolates the overall path from the intricate internal dynamics of human movement. Similarly, Zhanbo Huang et al. frame this as separating "world flow" (environmental movement) from "local flow" (subject-relative movement). Because subject-relative motion is insulated from environmental shifts, it provides far more discriminative, noise-free features for tasks like action analysis and gait recognition.
Why Disentanglement Works
Decoupling these streams provides powerful inductive biases that fundamentally change how models predict, synthesize, and control motion.
1. Simplifying Forecasting and Honoring Physical Laws
Predicting future trajectories becomes significantly easier when a model does not have to compute fine-grained limb oscillations to understand where an agent is traveling. Ruixuan Yu et al. utilize this decomposition to ensure geometric equivariance, simplifying the learning process by decoupling the overall translation of a system from its internal relative dynamics.
Furthermore, this separation aligns beautifully with classical physics. Haixu Wu et al. demonstrate that decomposing motion into immediate local variations and long-term global trends provides a strong inductive bias for modeling physical laws like inertia. This allows networks to accurately track both sudden, transient changes (like a sudden hand gesture) and consistent, long-term trajectories over time.
2. Stabilizing Multi-Agent and Long-Term Synthesis
In generative tasks, modeling absolute coordinates frequently causes long-term animations to dissolve or drift. By decoupling global spatial trajectories from local joint movements, generative frameworks can prioritize the semantic qualities of an interaction over raw coordinate transformations.
As Jogendra Nath Kundu et al. explain, isolating relative pose dynamics from an agent's position in the environment simplifies the learning of complex, multi-person interactions. This insight is echoed in frameworks like MUGL and DSAG, where fixing the root joint for local kinematic poses proves essential for scaling models to handle variable actor counts and complex, long-term locomotion without compounding spatial errors.
3. Overcoming Spatial Constraints in 3D/4D Generation
In modern text-to-4D generation, models often suffer from a "walk-in-place" limitation, where an asset performs an action but remains anchored to a static bounding box. Sherwin Bahmani et al. overcome this constraint by factorizing motion into global rigid transformations of a bounding box and local time-varying deformation fields. This allows generated 3D objects to traverse large-scale environments realistically while maintaining fine-grained, articulated animations like walking, flapping, or galloping.
Architectural Implementations
To operationalize this concept, modern architectures typically employ dual-branch networks or specialized attention mechanisms that process global and local features in parallel before fusing them.
Input Motion ──► Global Branch (Trajectory / Trends) ──┐
└── Local Branch (Joints / Deformations) ─┴─► Cross-Interaction Block ──► Prediction/Generation
For instance, the EMPMP framework utilizes lightweight dual branches to separate individual (local) and scene-level (global) motion dynamics, using a cross-level interaction block to capture holistic movement patterns at a fraction of the computational cost of standard Transformers.
In unsupervised learning, Boeun Kim et al. introduce a "Global-Local Attention" mechanism within their GL-Transformer to independently analyze whole-body displacement and fine-grained limb dynamics. This ensures that the model captures both the "big picture" of spatial navigation and the nuances of multi-person interactions.
This explicit separation also offers a robust defense against external noise. In camera-controlled video generation, Zhijing Cheng et al. decouple global camera-induced motion from local object-intrinsic dynamics. By processing these streams independently, their framework prevents complex camera movements from warping or conflicting with the natural physical actions of the objects within the scene.
Ultimately, global-local motion disentanglement transforms a highly complex, entangled spatial problem into two simpler, specialized sub-problems. By treating "where things go" and "how things move" as distinct but interacting phenomena, models achieve unprecedented stability, realism, and efficiency across tracking, prediction, and synthesis.
Key Foundational Work
Multiple Granularity Group Interaction Prediction (2018) — contributed 1 concept elements to this direction.
3D People Tracking with Gaussian Process Dynamical Models (2006) — contributed 1 concept elements to this direction.