Sequential View Grasp Detection For Inexpensive Robotic Arms

Marcus Gualtieri, Robert Platt Jr.

Abstract

Grasp detection in robotics is often hampered by poor sensor data resulting from suboptimal viewpoints, leading to occlusions and noise. This paper presents an active vision approach that strategically selects a second viewpoint based on object geometry. By mapping grasp detection performance across a hemisphere of perspectives, the proposed system can detect up to six times more valid grasps and improve overall grasping success rates by 12% on inexpensive robotic hardware.

Introduction

Identifying valid hand configurations for grasping—known as grasp detection—is highly sensitive to the quality of visual sensor input. Most contemporary systems rely on a "one-shot" approach from an arbitrary initial viewpoint. However, common depth sensors like the Kinect or RealSense are prone to noise, perspective distortions, and occlusions that can hide viable grasp points. This paper addresses the "viewpoint selection problem": determining which camera position maximizes the density and confidence of detected grasps for a target object.

Methodology

The authors propose a two-step active vision framework to refine grasp detection through strategic movement.

1. Performance Mapping

Using the BigBIRD dataset, the researchers empirically mapped how grasp detection performance varies based on the camera's azimuth ($\theta$) and elevation ($\phi$) relative to the object. These maps were categorized by geometry, specifically for box-like and cylinder-like objects, to identify "sweet spots" for sensor placement.

2. Smart Viewpoint Selection

A selection strategy was developed to move the camera to a viewpoint that maximizes the expected number of True Positives minus False Positives. The objective is to achieve a high density of valid grasp candidates, providing the robot with multiple options to satisfy its internal kinematic constraints.

The robotic hardware setup (a) and the hemispherical viewpoint mapping system (b).

3. Alignment View

To mitigate mechanical errors such as kinematic calibration drift, an "alignment" step was introduced. Once a high-quality grasp is identified from the smart viewpoint, the robot takes a final "head-on" look to precisely align the gripper with the point cloud before execution.

Key Insights: Quantity is Quality

A primary finding of this research is that the viewpoint significantly impacts the number of detected grasps more than the raw classification accuracy of any single candidate. By selecting an optimal viewpoint, the robot can generate a much denser set of potential grasps. This increased density is crucial because it allows the robot's Inverse Kinematics (IK) solvers to find a path that is both valid and reachable within the robot's physical workspace.

Grasp detection evolution: (a) initial point cloud, (b) dense candidate generation from an optimal viewpoint, and (c) filtered valid grasps.

Experimental Results

The system was evaluated using a Baxter research robot tasked with clearing cluttered objects. The results demonstrated significant performance gains:

  • Detection Density: The smart viewpoint selection increased the number of detected true positive grasps by a factor of 6 compared to random viewpoints.
  • Success Rate:
    • Smart viewpoint selection alone provided a 4% boost.
  • The "head-on" alignment strategy provided an 8% boost.
  • The combined strategy improved the baseline grasp success rate from 75% to 87%, a total improvement of 12%.

Conclusion and Future Work

This work demonstrates that active vision can significantly compensate for the limitations of inexpensive sensors and robotic arms. While currently limited to coarse geometric categories (boxes and cylinders), future iterations could benefit from more granular object classification or continuous viewing trajectories. Additionally, integrating reachability constraints directly into the viewpoint selection process would further optimize the robot's ability to execute grasps in complex, cluttered environments.

Summary updated

BibTeX
@inproceedings{gualtieri2017sequential,
  author = {Marcus Gualtieri and Robert Platt Jr.},
  title = {{Sequential View Grasp Detection For Inexpensive Robotic Arms}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2017},
  doi = {10.1109/iros.2017.8202166},
}
Figure