GAP-RL: Grasps as Points for RL Towards Dynamic Object Grasping

Pengwei Xie, Siang Chen, Qianrun Chen, Wei Tang, Dingchang Hu, Yixiang Dai, Rui Chen, Guijin Wang

TL;DR

This paper introduces GAP-RL, a novel reinforcement learning framework for dynamic object grasping. By representing grasps as Gaussian points and using a hierarchical encoder, GAP-RL achieves superior generalization to novel objects and unseen motions, demonstrating strong sim-to-real transferability.

What Problem Does This Paper Address?

Grasping moving objects reliably is a significant challenge in robotics. Existing methods often struggle with dynamic scenarios, requiring precise object tracking, motion prediction, or relying on pre-generated grasps that may not adapt well to novel objects or unpredictable movements. While reinforcement learning (RL) offers a promising closed-loop approach, current RL methods often fail to fully leverage visual information for enhanced grasp performance, especially in dynamic settings. This paper aims to bridge this gap by proposing a new way to represent and learn grasp policies for dynamic objects.

Methods

The core of GAP-RL is its novel grasp representation and encoding strategy. Instead of using traditional 6D pose parameters or keypoints, grasps are transformed into a set of random Gaussian points. These points are then fed into a hierarchical grasp encoder called GraspGroupNet (GGN). GGN first captures intra-grasp details and then aggregates these features to analyze inter-grasp relationships, creating a rich, abstract representation of the grasp.

To facilitate this, the framework includes a Graspable Region Explorer that efficiently identifies potential graspable areas on the target object, even in the presence of motion blur, by considering adjacent regions. A fast, region-based grasp detector (Local Grasp) then generates candidate grasps. The RL policy, trained using the Soft Actor-Critic (SAC) algorithm, takes these encoded grasp features along with robot state information to predict a 6-DoF residual adjustment for the robot arm's Cartesian pose and a 1-DoF action for gripper control. The reward function is shaped to guide the agent through approaching, grasping, and lifting stages, incorporating auxiliary tasks for improved performance.

Datasets

  • Simulation Benchmark: The authors construct a simulated dynamic grasping benchmark using the SAPIEN simulator. This benchmark features objects undergoing diverse dynamic motions, including rotation, linear movement, circular trajectories, and random Bezier curves, at varying speeds.
  • Object Datasets:
    • Graspnet-1billion Dataset: Used for training and evaluating on "Seen Split" and "Unseen Split" object sets.
  • Acronym Dataset: Used for evaluating on "Novel Split" object sets, representing novel items.

Key Insights and Ideas

  • Grasps as Points: Representing grasps as sets of Gaussian points, rather than explicit poses or keypoints, allows for a more flexible and generalizable representation that is less prone to overfitting to specific gripper geometries.
  • Hierarchical Grasp Encoding (GGN): The two-stage encoding process (intra-grasp details followed by inter-grasp relationships) is crucial for extracting robust and informative features from the grasp point representations.
  • Graspable Region Explorer: This module is vital for handling real-world challenges like motion blur, enabling more stable grasp detection by focusing on consistent regions.
  • Sim-to-Real Transferability: The proposed grasp representation and encoding method significantly improve the transferability of policies trained in simulation to real-world robotic systems, as evidenced by the qualitative results showing indistinguishable feature distributions.

Key Results

  • Simulation Performance: GAP-RL significantly outperforms baseline methods (Heuristic Planning, GAMMA, EARL, GA-DDPG) across various settings (Seen/Unseen Split, Seen/Novel Traj.), achieving success rates up to 18% higher than the second-best method for seen objects and 34% for novel objects in the "Novel Traj." setting.
  • Real-World Performance: On real robot experiments, GAP-RL achieved an average success rate of 76.3%, outperforming GA-DDPG (57.9%) and Heuristic Planning (42.9%) across different dynamic scenarios (Conveyor, Turntable, Combined, Handover).
  • Ablation Studies: Replacing Gaussian Points with keypoints or using a simpler PointNet encoder for grasp features led to a significant drop in performance, highlighting the effectiveness of GAP-RL's proposed components.
  • Qualitative Results: t-SNE visualizations show that GAP-RL's learned features from simulated depth images are indistinguishable from those derived from real-world depth images, demonstrating superior sim-to-real transferability.

Limitations and Future Work

  • Single-Frame Input: The current methodology relies on single-frame depth input, which can lead to failures due to collisions if the object moves and slips unexpectedly. The authors suggest incorporating temporal association mechanisms for improved performance.
  • Single-Object Scenarios: The framework is designed for single-object grasping and cannot handle cluttered scenes or scenarios with multiple interfering objects. Future work could involve integrating segmentation modules to isolate target objects in more complex environments.
  • Control Frequency: Performance degrades at higher object velocities due to limitations in control frequency and the robotic arm's speed.
  • Future Directions: The authors plan to explore the integration of temporal association information to enrich grasp representation and optimize multi-frame policies.

Summary updated

Abstract

Dynamic grasping of moving objects in complex, continuous motion scenarios remains challenging. Reinforcement Learning (RL) has been applied in various robotic manipulation tasks, benefiting from its closed-loop property. However, existing RL-based methods do not fully explore the potential for enhancing visual representations. In this letter, we propose a novel framework called Grasps As Points for RL (GAP-RL) to effectively and reliably grasp moving objects. By implementing a fast region-based grasp detector, we build a Grasp Encoder by transforming 6D grasp poses into Gaussian points and extracting grasp features as a higher-level abstraction than the original object point features. Additionally, we develop a Graspable Region Explorer for real-world deployment, which searches for consistent graspable regions, enabling smoother grasp generation and stable policy execution. To assess the performance fairly, we construct a simulated dynamic grasping benchmark involving objects with various complex motions. Experiment results demonstrate that our method effectively generalizes to novel objects and unseen dynamic motions compared to other baselines. Real-world experiments further validate the framework's sim-to-real transferability.
BibTeX
@article{xie2025gap,
  author = {Pengwei Xie and Siang Chen and Qianrun Chen and Wei Tang and Dingchang Hu and Yixiang Dai and Rui Chen and Guijin Wang},
  title = {{GAP-RL: Grasps as Points for RL Towards Dynamic Object Grasping}},
  journal = {IEEE RA-L},
  year = {2025},
  doi = {10.1109/lra.2024.3495333},
}
Figure