GMAP: Generalized Manipulation of Articulated Objects in Robotic Using Pre-trained Model

Hongliang Zeng, Ping Zhang, Fang Li, Qinpeng Yi, Tingyu Ye, Jiahua WangAAAI 2025 · DOI

Abstract

GMAP is a comprehensive "perception-to-interaction" framework designed to enable robots to manipulate articulated objects such as doors, drawers, and laptops. While high-level reasoning is increasingly handled by Large Language Models, precise physical coordination remains a significant challenge. By combining a multi-scale pre-trained point cloud encoder with precise joint estimation and affordance mapping, GMAP allows robots to plan and execute complex trajectories from simple high-level commands. Evaluated on PartNet-Mobility and in SAPIEN simulations, the framework achieves state-of-the-art performance in part segmentation and manipulation success rates, demonstrating robust generalization across unseen object categories.

Introduction

Articulated objects present a unique challenge for robotics due to their specific kinematic constraints—for example, a door swings on a hinge, while a drawer slides on a rail. While Large Language Models (LLMs) excel at high-level task planning (the "brain"), robots often lack the "cerebellum" required for precise physical interaction. Existing research typically addresses part segmentation, joint axis estimation, and affordance prediction in isolation. GMAP aims to bridge these gaps into a unified system that generalizes across different object categories even when annotated data is scarce.

Methods

The GMAP framework utilizes a multi-stage pipeline to transform raw point clouds into actionable manipulation plans.

The GMAP architecture featuring Multi-Scale Feature Extraction (MSFE), Point-wise Feature Extraction (PFE), and specialized downstream networks.

Multi-Scale Feature Extraction (MSFE)

To handle objects of various sizes, the system partitions point clouds into multiple scales and employs a Vision Transformer (ViT) encoder. This module is pre-trained using self-supervised learning (Point-MGE) on the ShapeNet dataset, allowing the model to learn general 3D geometric features before being fine-tuned for articulated tasks.

Specialized Downstream Networks

  • Seg-Net: Performs part-level segmentation and identifies movable components.
  • Para-Net: Estimates kinematic parameters, including joint type (revolute vs. prismatic), orientation, position, and the current state of the object.
  • Afford-Net: Predicts action proposals (where to grab) and action scores (likelihood of success).

Dynamic Trajectory Planning

Unlike baseline methods that rely on static motion primitives, GMAP uses estimated joint parameters to calculate precise trajectories in SE(3) space. This ensures the robot's end-effector follows the natural kinematic arc or line of the object's movement.

Experimental Evaluation

The authors utilized several key benchmarks for training and validation:

  • ShapeNet: Used for large-scale self-supervised pre-training (50,000+ models).
  • PartNet-Mobility: The primary dataset for evaluating articulated part segmentation and joint estimation.
  • SAPIEN: A physics simulator used to test manipulation success rates.

Results and Key Insights

GMAP achieved state-of-the-art (SOTA) results across several metrics. In perception tasks, it reached an 80% Intersection over Union (IoU) in part segmentation and reduced joint orientation error to just 0.42°. In the SAPIEN simulator, GMAP reached a 36.94% success rate for instruction-based pushing, significantly outperforming previous benchmarks like AdaAfford (27.71%).

Qualitative results showing segmentation, affordance mapping, and joint estimation across various articulated object categories.

Key insights from the study include:

  • Pre-training Efficacy: Pre-training on general 3D shapes significantly boosts performance on specialized articulated tasks by providing a better understanding of 3D structure.
  • Multi-Scale Necessity: Capturing both local features (like hinges) and global features (like doors) is vital for precise joint estimation.
  • Generalization: When trained on cabinets and tested on unseen categories like ovens and fridges, the model maintained high accuracy, proving it learned general principles of articulation.

Conclusion and Future Work

While GMAP demonstrates robust performance, the authors identified areas for future refinement, including addressing segmentation artifacts in tightly packed objects and improving the accuracy of initial interaction points. Future work will also explore multi-joint articulated objects, such as folding lamps or multi-link robotic arms, to further expand the framework's versatility.

Summary updated

BibTeX
@inproceedings{zeng2025gmap,
  author = {Hongliang Zeng and Ping Zhang and Fang Li and Qinpeng Yi and Tingyu Ye and Jiahua Wang},
  title = {{GMAP: Generalized Manipulation of Articulated Objects in Robotic Using Pre-trained Model}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i14.33615},
}
Figure