Traditional computer vision and robotics often represent human motion as a sequence of coordinates in Euclidean space. However, the human body is not a collection of independent points moving in flat space; it is a system of articulated joints governed by physical constraints, fixed bone lengths, and rotational limits. When we treat these movements as simple vectors, we encounter mathematical artifacts like gimbal lock, discontinuities in periodic angles, and "ghosting" during averaging.
To address this, researchers have shifted toward modeling motion on Riemannian manifolds—curved mathematical surfaces that naturally encode the geometric constraints of the body.
The Geometry of a Pose
In this framework, a single body pose is not just a list of numbers, but a point on a specific manifold. For example, Natola et al. and Xu et al. represent joint poses as transformation matrices within the Special Euclidean group SE(3). This captures both rotation and translation while ensuring that the physical structure of the skeleton—such as the distance between the shoulder and elbow—remains constant.
Other researchers, such as Pala et al. and Brun, represent motion using Symmetric Positive Definite (SPD) matrices. By modeling joint sequences as covariance descriptors on an SPD manifold, the authors capture the non-linear spatial and temporal correlations between different body parts. This approach is particularly effective for recognizing subtle behavioral signals, such as emotion or pain, because it respects the underlying geometry of how joints move in relation to one another.
Actions as Trajectories
If a pose is a point on a manifold, then an action—like walking, jumping, or waving—is a continuous curve, or trajectory, across that manifold. Anirudh et al. demonstrate that treating entire video sequences as single paths allows for a massive reduction in dimensionality while remaining invariant to timing differences.
This perspective changes how we compare two movements. In flat Euclidean space, the "average" of two rotations might result in a physically impossible pose. On a manifold, we use geodesics—the shortest path between two points on a curved surface. Daab et al. show that calculating these geodesics preserves orientation constraints, leading to smoother and more accurate robot motions when learning from human demonstrations. Similarly, Meng et al. use the Karcher Mean—a geometric average on the manifold—to create archetypal templates for actions, allowing a system to recognize a "drinking" motion regardless of the speed at which it is performed.
Bridging Manifolds and Deep Learning
While manifolds offer a superior representation, most machine learning tools are designed for flat Euclidean data. To bridge this gap, researchers use the Logarithmic Map to project manifold data onto a tangent space—a local, flat approximation of the manifold at a specific point.
Sun et al. utilize this projection to apply linear dynamical systems to non-Euclidean orientation data, avoiding singularities like gimbal lock. For deep learning, Huang et al. introduced LieNet, an architecture that processes skeletal data directly on Lie groups. By keeping the data on the manifold for as long as possible, these models preserve the structural fidelity of the motion, leading to higher accuracy in action recognition compared to models that flatten the data into standard vectors.
Applications in Health and Robotics
This geometric approach has profound implications for specialized motion analysis:
- Clinical Diagnostics: Manzanera et al. use Riemannian geometry to classify Parkinson’s Disease patterns, capturing subtle pathological variations that linear models might obscure. Szczapa et al. apply similar logic to facial landmarks to estimate pain intensity.
- Physical Therapy: Devanne et al. represent rehabilitation exercises as distributions on manifolds, providing a probabilistic framework to identify errors in a patient's form.
- Robot Retargeting: Klein et al. argue that human motion planning is intrinsically linked to the body's mass-inertia matrix. By modeling motion on a manifold defined by these physical dynamics, they more accurately "retarget" human movements onto humanoid robots.
By treating the body as a geometric entity rather than a set of coordinates, these methods ensure that the mathematical representation of motion is as constrained and fluid as the physical movement it describes.
Key Foundational Work
Synergistic Face Detection and Pose Estimation with Energy-Based Models (2007) — contributed 2 concept elements to this direction.
Regressed Importance Sampling on Manifolds for Efficient Object Tracking (2009) — contributed 1 concept elements to this direction.