GEAR: GEometry-Motion Alternating Refinement for Articulated Object Modeling with Gaussian Splatting

Jialin Li, Bin Fu, Ruiping Wang, Xilin Chen

TL;DR

Reconstructing articulated objects from images is a classic chicken-and-egg problem: you need accurate part segmentation to estimate motion, but you need motion cues to segment the parts. GEAR solves this by using an Expectation-Maximization (EM)-style alternating optimization framework built on Gaussian Splatting, decoupling geometry and motion to achieve stable, high-fidelity reconstructions of complex multi-joint objects.

What Problem Does This Paper Address?

Creating interactive digital twins of articulated objects—like cabinets, scissors, or laptops—is crucial for robotics, virtual reality, and embodied AI. However, modeling them from multi-view images is incredibly difficult because geometry and motion are deeply coupled.

If you try to optimize both at once (coupled optimization), the model gets stuck in local minima, often distorting the 3D geometry to compensate for bad motion estimates. If you do it in stages (staged optimization), early segmentation errors permanently ruin the final motion parameters. Furthermore, existing methods often rely on category-specific fine-tuning, which fails to generalize to novel or complex multi-joint objects.

Figure 1. Articulated object modeling involves coupled optimiza- tion of geometry and motion. Motion analysis plays a key role in part reconstruction, while joint motion parameter estimation relies on accurate part segmentation. Unlike coupled and staged methods, GEAR utilizes alternating optimization to improve stability.

Methods

GEAR uses 2D Gaussian Splatting (2DGS) as its core representation and structures the reconstruction as a three-stage coarse-to-fine process:

  1. Initialization: A voxel-based coarse module compares voxel occupancy between the object's states (e.g., open vs. closed) to identify dynamic regions and provide a stable starting point for masks and motion parameters.
  2. E-step (Geometry Modeling): While freezing motion parameters, GEAR updates the part assignment probabilities for each Gaussian. To ensure sharp boundaries and multi-view consistency without sacrificing generalization, the authors introduce a SAM Mask Aggregation module. It takes fine-grained, inconsistent 2D masks from a vanilla Segment Anything Model (SAM) and aggregates them into consistent 3D part-level masks using a voting-based strategy.
  3. M-step (Motion Modeling): While freezing the geometry masks, GEAR optimizes the rigid body motion parameters (represented via dual quaternions). Crucially, it binarizes the soft masks into hard assignments during this step to enforce strict physical rigid-body constraints and avoid geometric distortions.
Figure 2. GEAR is an EM-style framework with three modules: Initialization, geometry modeling, and motion modeling. The key idea is the alternating optimization of geometry and motion, which enhances the stability and performance of articulated object modeling.

Datasets

The authors evaluate GEAR on three benchmarks:

  • PARIS: A standard dataset containing 12 single-joint objects (10 synthetic, 2 real-world).
  • ArtGS-Multi: A dataset with 5 multi-joint objects.
  • GEAR-Multi (New): Introduced by the authors to address the lack of complex benchmarks, this dataset contains 10 diverse multi-joint objects across 10 categories (sourced from the SAPIEN/PM dataset), specifically targeting objects with multiple movable parts (e.g., storage units with multiple drawers and doors).

Key Insights and Ideas

  • Breaking the Loop with EM: Treating part segmentation as a latent variable and motion parameters as explicit model parameters allows the framework to alternate between refining geometry and motion, preventing error propagation.
  • Weak Supervision over Fine-Tuning: Instead of fine-tuning SAM on specific object categories (which hurts zero-shot generalization), GEAR uses vanilla SAM as a weak supervisor. The aggregation mechanism filters out SAM's multi-view inconsistencies.
  • Hard Assignments for Physical Rigidity: While soft blending is great for smooth gradient flow during mask optimization (E-step), enforcing hard assignments during motion optimization (M-step) is essential to prevent non-physical warping of the underlying 3D geometry.

Key Results

GEAR achieves state-of-the-art performance across all benchmarks, particularly shining on complex multi-joint objects.

  • Superior Accuracy: On the new GEAR-Multi dataset, GEAR significantly outperforms the previous state-of-the-art (ArtGS), reducing the average joint axis angular error from 8.82° to 0.09° and the Chamfer Distance for moving parts (CD-m) from 115.70 to 0.71.
  • Convergence Stability: Joint optimization curves oscillate heavily without converging, and staged optimization quickly stagnates. In contrast, GEAR's alternating strategy steadily drives down both motion error and training loss.
  • Efficiency: The framework converges quickly; when scaled down to match baseline iterations, it achieves superior accuracy in just ~11 minutes per object.
Figure 6. Convergence Analysis on Storage 45271. (a) Sum of motion parameter errors (Axis Angle + Position + Geometry Distance) over iterations. (b) Total training loss over iterations. Our alternating optimization (blue) achieves the lowest error and loss, avoiding local minima traps and error accumulation.

Limitations and Future Work

  • Extreme Articulations: GEAR struggles with very large state transitions, such as 180-degree rotations, where the initialization step might fail to find proper correspondences.
  • Challenging Materials: Reconstructing transparent or highly reflective materials remains difficult for the underlying Gaussian Splatting representation.
  • Future Directions: The authors suggest integrating explicit motion priors to handle extreme cases and extending the representation to capture broader physical and material properties.

Summary updated

Abstract

High-fidelity interactive digital assets are essential for embodied intelligence and robotic interaction, yet articulated objects remain challenging to reconstruct due to their complex structures and coupled geometry-motion relationships. Existing methods suffer from instability in geometry-motion joint optimization, while their generalization remains limited on complex multi-joint or out-of-distribution objects. To address these challenges, we propose GEAR, an EM-style alternating optimization framework that jointly models geometry and motion as interdependent components within a Gaussian Splatting representation. GEAR treats part segmentation as a latent variable and joint motion parameters as explicit variables, alternately refining them for improved convergence and geometric-motion consistency. To enhance part segmentation quality without sacrificing generalization, we leverage a vanilla 2D segmentation model to provide multi-view part priors, and employ a weakly supervised constraint to regularize the latent variable. Experiments on multiple benchmarks and our newly constructed dataset GEAR-Multi demonstrate that GEAR achieves state-of-the-art results in geometric reconstruction and motion parameters estimation, particularly on complex articulated objects with multiple movable parts. Project Page: https://github.com/VIPL-VSU/GEAR.
BibTeX
@inproceedings{li2026gear,
  author = {Jialin Li and Bin Fu and Ruiping Wang and Xilin Chen},
  title = {{GEAR: GEometry-Motion Alternating Refinement for Articulated Object Modeling with Gaussian Splatting}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2026},
  url = {https://openreview.net/forum?id=Le6RPE5GfK},
}

Loading related content...

Figure