Introduction
Robotic pick-and-place systems often struggle in unstructured environments because they rely on "guessed" geometry. When a robot only sees the front of an object, it must use shape completion to estimate the hidden parts. Standard motion planners typically treat these estimates as ground truth, leading to failures when the guess is wrong—such as attempting to grasp a non-existent handle. This paper introduces a modular pipeline that explicitly incorporates perceptual uncertainty into a regrasp planner’s cost function, allowing the robot to choose safer, more reliable actions.
System Architecture
The authors propose a three-level planning pipeline designed to bridge the gap between perception and action. The process begins with an Instance Segmentation module (BoNet) and a Shape Completion module (modified PCN). The output of these modules informs an Arrangement Planner and a Regrasp Planner, which together determine the sequence of moves required to achieve a goal state. Finally, a Motion Planner generates the specific trajectories for the robot arm.
Uncertainty-Aware Perception and Planning
A key innovation is the modification of the Point Completion Network (PCN). The authors added a second decoder to estimate the probability that each completed point is accurate. This allows the system to distinguish between high-confidence observed surfaces and low-confidence "hallucinated" geometry.
To utilize this information, the regrasp planner uses an $A^*$ search with a cost function that maximizes the joint probability of success for every pick and place move. Several uncertainty modeling methods were evaluated:
- Grasp Quality (GQ): A standard geometric heuristic.
- Monte-Carlo (MC): Sampling multiple possible shapes (accurate but computationally expensive).
- Contact Uncertainty (CU): Penalizing grasps on low-confidence points.
- Success Prediction (SP): A dedicated neural network that predicts the probability of a stable grasp directly from the completed point cloud and gripper pose.
Key Insights and Results
The research highlights that perception is the primary bottleneck in complex manipulation; moving from ground-truth to estimated geometry caused a performance drop of up to 22% in baseline planners. However, the Success Prediction (SP) method significantly mitigated this, outperforming other heuristics by considering the global distribution of mass and shape rather than just local contact points.
Experimental Performance
- Real-World Packing: In experiments with a UR5 robot and 34 novel objects, the SP-based cost function achieved a 7.8% higher success rate than standard planners.
- Simulation Success: In canonical placement tasks, the SP method reached a 91.3% success rate, compared to 72.7% for uncertainty-unaware systems.
- Risk-Aversion: The system naturally favors "regrasps"—repositioning an object to a more stable orientation—when the initial geometry is too uncertain to guarantee a safe one-shot move.
Conclusion and Future Work
By making robots "aware" of their own perceptual doubt, this work demonstrates a significant improvement in the reliability of robotic manipulation. While the current system is passive, future research will explore active vision, where the robot moves the camera to proactively reduce uncertainty. Additionally, improving the speed of the regrasp planner and expanding the diversity of training data for shape completion remain key areas for development.
Summary updated