Introduction
Grasping a specific object in a crowded space—such as picking a single key from a bowl or a specific component from a bin—presents a significant challenge for robotic systems. In these "Singulation-Necessary Clutter" (SNC) environments, the target object is often physically ungraspable because it is tightly surrounded by obstacles.
Traditional robotic approaches typically rely on two-fingered grippers, which lack the dexterity to manipulate objects in tight spaces without moving the entire arm, often leading to collisions. Alternatively, multi-stage policies treat "clearing the area" (singulation) and "grasping" as separate, sequential tasks. This is often inefficient, as the robot might fail to capitalize on fleeting opportunities to grab the object during the clearing phase. DexSinGrasp addresses these limitations by creating a single, unified policy that leverages the high degrees of freedom (DoF) of a dexterous hand to perform subtle finger maneuvers that separate and grasp objects in one fluid motion.
Methodology
The DexSinGrasp framework is built upon three core technical components designed to handle the complexity of dense clutter.
Unified Reward Design
Instead of a state machine that switches modes, DexSinGrasp utilizes a piece-wise reward function. This function incentivizes the hand to approach the target and separate it from neighbors simultaneously. Once a "contact criterion" is met, the reward shifts to prioritize lifting and goal-reaching, ensuring a smooth transition from singulation to grasping without discrete task switching.
Clutter Arrangement Curriculum Learning
Learning to grasp in dense clutter from scratch is a "sparse reward" problem. The authors introduced a curriculum that starts with a single object and progressively adds complexity: moving to 4, 6, and 8 objects in Dense Arrangements (tightly packed), and finally progressing to Random Arrangements. Training on dense, structured clutter first was found to provide a superior foundation for generalizing to random clutter.
Teacher-Student Policy Distillation
To enable real-world deployment, the system uses a distillation process:
- Teacher Policy: Trained in simulation using "privileged" information, such as exact object poses and distances.
- Student Policy: A vision-based policy that only perceives point clouds. It uses a pre-trained encoder (UniGraspTransformer) to process visual data and is trained via behavior cloning to mimic the Teacher. This allows the system to operate in environments where perfect state data is unavailable.
Experimental Results
The authors evaluated DexSinGrasp using a new benchmark for Singulation-Necessary Clutter (SNC), including dense, random, and irregular clutter sets.
- High Success Rates: In the most challenging simulated environments (D-8), DexSinGrasp achieved a 97% success rate. In contrast, a baseline policy that only rewarded grasping (without explicit singulation rewards) managed only a 10% success rate.
- Efficiency and Speed: The unified policy proved significantly faster than multi-stage baselines, as it begins the grasping motion as soon as a sufficient gap is created, rather than waiting for full clearance.
- Emergent Behaviors: An interesting emergent behavior was "finger flickering," where the policy used rapid, subtle finger vibrations to displace surrounding objects without moving the palm—a maneuver impossible for standard parallel-jaw grippers.
- Real-World Transfer: When deployed on a physical uFactory xArm6 with a LEAP hand, the vision-based student policy achieved a 61% success rate on "practical clutter" (random household objects) with zero-shot transfer.
Limitations and Future Work
While DexSinGrasp demonstrates high performance in tabletop scenarios, it currently focuses on single-layer 2D clutter. Future work will aim to address "3D clutter," where objects are stacked or buried. Additionally, while the policy handled irregular 2D shapes well, highly complex 3D topologies remain a challenge. Finally, integrating tactile sensing could improve robustness in scenarios where the target object is occluded by obstacles.
Summary updated