Introduction
Motion in the physical world is rarely a collection of independent, stochastic movements. Instead, it is inherently structured and hierarchical: a finger’s trajectory is defined relative to the hand, which in turn moves relative to the arm. Capturing these "parent-child" relationships is essential for high-fidelity computer vision, robotics, and graphics.
Historically, researchers have approached motion modeling through two primary lenses. The first involves manual templates, such as pre-defined skeletal rigs. While effective for human characters, these templates fail to generalize to arbitrary objects like excavators, folding fabrics, or fluid splashes. The second approach utilizes black-box neural networks to predict motion fields directly. While flexible, these models often lack interpretability and frequently produce "rubbery" or physically implausible distortions because they do not account for underlying structural constraints.
HEIR (Hierarchical Motion Learning) introduces a novel framework designed to automatically discover these hierarchies directly from raw trajectory data, requiring no prior knowledge of an object’s category or shape.
The HEIR Framework
HEIR treats motion elements—such as points in space or 3D Gaussian splats—as nodes within a graph. The framework formulates motion decomposition as a differentiable graph learning problem through three core components:
1. The Learnable Motion Graph (Encoder)
HEIR utilizes a Graph Neural Network (GNN) to analyze the absolute velocities of all points. By employing an attention mechanism, the encoder predicts "edge weights" that represent the probability of one node being the parent of another. This transforms the structural discovery process into a continuous optimization problem.
2. Differentiable Sampling
To convert these probabilities into a concrete, discrete hierarchy (specifically a Directed Acyclic Graph), the authors utilize the Gumbel-Softmax trick. This allows the model to select a specific parent for each node while maintaining end-to-end differentiability, enabling the system to learn the optimal structure through backpropagation.
3. The Kinematic Decoder (Neumann Series)
A key innovation of HEIR is its use of a truncated Neumann series to reconstruct total motion. In a hierarchy, a point's global motion is the sum of its local motion and the motion of all its ancestors. The Neumann series effectively "unrolls" the tree structure, allowing the model to calculate how global motion trickles down to local parts efficiently. To ensure the hierarchy is meaningful, the authors apply an $\ell_1$ penalty to the relative motion. This forces the model to explain as much movement as possible through the hierarchy, leaving only minimal residuals for local adjustments.
Experimental Evaluation
The authors evaluated HEIR across three distinct scales to test its robustness and accuracy:
- 1D Toy Example: A synthetic dataset of 11 nodes moving in nested sine waves. HEIR successfully recovered the ground-truth hierarchy in 73% of trials, whereas random chance would yield a success rate of effectively zero ($5.0 \times 10^{-7}\%$).
- 2D Planetary System: A "star-planet-moon" simulation designed to test the model’s ability to handle rotational inheritance and complex orbital paths.
- D-NeRF Benchmark: A standard for dynamic 3D scene reconstruction. Testing on complex scenes like the Excavator, Hook, and Warrior, HEIR was integrated with 3D Gaussian Splatting to evaluate its performance on thousands of points.
Key Results and Insights
Quantitative Superiority
In the D-NeRF Excavator scene, HEIR significantly outperformed the state-of-the-art SC-GS baseline. It achieved a PSNR of 21.56 (compared to 19.91) and a substantially lower LPIPS (perceptual error) of 0.0383 (compared to 0.065). These metrics indicate that the hierarchical approach leads to much more accurate reconstructions of dynamic scenes.
Structural Rigidity and Interpretability
Qualitatively, HEIR preserves the physical integrity of objects. While baseline methods often cause rigid parts (like a metal excavator arm) to stretch or bend unnaturally, HEIR correctly identifies these parts as rigid units following a parent joint. Furthermore, because HEIR produces an explicit graph, the resulting model is highly interpretable. Users can inspect the discovered hierarchy and even manually manipulate "handles" within the graph to control the animation.
Limitations and Future Directions
Despite its success, HEIR faces several challenges:
- The Static Problem: The model can only infer relationships from observed motion. If a component (e.g., a shovel on a parked truck) remains stationary during training, its hierarchical position cannot be determined.
- Single-Parent Constraint: The current model assumes a tree structure where each node has one parent. Future iterations may need to support Directed Acyclic Graphs with multiple parents to model complex constraints like mechanical loops.
- Long-Range Dependencies: Currently, the model relies on $k$-nearest neighbors for its initial graph. Moving toward global attention mechanisms could help capture correlations between spatially distant but motion-linked parts.
Summary updated