Abstract
Modern robotic grasping systems often struggle with task-specific requirements, such as picking up a tool by its handle rather than its functional end. We present the Variational Constrained Grasp Sampler (VCGS), a deep generative model that enables robots to focus grasping attempts on specific, user-defined areas of an object. By training on the massive new CONG dataset, VCGS achieves a 75% success rate in real-world trials—a 15% improvement over state-of-the-art unconstrained samplers—while being up to three times more sample-efficient.
Introduction
Most modern robotic grasping research focuses on "unconstrained" sampling: finding any stable, collision-free grasp on an object. While effective for simple pick-and-place tasks, this approach fails when specific parts of an object must be targeted for functional reasons, such as grasping a bottle by its body to squeeze it or a screwdriver by its handle to use it.
Existing solutions typically rely on inefficient filtering—generating thousands of random grasps and discarding those outside the target area—or rigid semantic models that only recognize specific categories like "handles." VCGS offers a general-purpose, task-agnostic alternative that can focus on any area designated by a user or a high-level planner.
Methodology
The core of the system is a Conditional Variational Autoencoder (CVAE) built on a PointNet++ backbone. Unlike standard samplers that only process the object's geometry, VCGS accepts the point cloud along with a point-wise binary feature. This feature acts as a spatial "highlight," informing the network which points belong to the target area.
A key insight of this research is that providing the entire object point cloud (with the target area highlighted) is superior to providing only the target area. The global context allows the robot to understand the object's overall geometry and avoid collisions with non-target parts. During inference, the model draws samples from a latent space and decodes them into 6-Degree of Freedom (6-DoF) grasp poses conditioned on the highlighted region.
The CONG Dataset
To train such a model, the authors introduced CONG (CONstrained Grasping), a dataset of unprecedented scale for this field. While previous constrained datasets were limited to a few hundred objects and specific tasks, CONG contains over 14 million training samples across 2,889 unique objects. It was constructed by expanding the ACRONYM dataset using a query-point method to identify all valid grasps within randomly sampled radii on object surfaces.
Experimental Results
The researchers benchmarked VCGS against GraspNet, a leading unconstrained sampler, using both Isaac Gym simulations and a physical Franka Emika Panda robot.
- Success Rate: In real-world tests, VCGS achieved a 75% success rate, significantly higher than the baseline's 59.4%.
- Efficiency: VCGS proved to be 1.8x to 3x more sample-efficient. It required an average of 360 samples to find a viable grasp, compared to 660 for the baseline.
- Inference Speed: By generating high-quality, constrained grasps directly, the system avoids the heavy computational cost of generating and then filtering tens of thousands of unconstrained candidates.
Conclusion and Future Work
VCGS demonstrates that incorporating spatial constraints directly into the generative process leads to more reliable and efficient robotic manipulation. Current limitations include a lack of explicit orientation constraints (e.g., "grasp from the side") and the potential for the sampler to propose grasps that are kinematically unreachable for specific robot arms. Future iterations aim to integrate robot kinematic limits and orientation-based constraints directly into the sampling architecture.
Summary updated