Viewpoint selection for grasp detection

Marcus Gualtieri, Robert Platt Jr.

TL;DR

This paper tackles the problem of improving grasp detection accuracy by intelligently selecting the robot's viewpoint. By analyzing how different viewpoints affect grasp detection performance, the authors develop a "smart" viewpoint selection strategy that significantly increases the number of detected grasps and their confidence, leading to higher real-world grasp success rates.

What Problem Does This Paper Address?

Grasp detection aims to identify potential grasping configurations directly from sensor data, without relying on explicit object models. While effective for novel objects, its performance can be highly sensitive to the quality and quantity of sensor data. This paper addresses the challenge of optimizing this sensor data acquisition by exploring the relationship between the robot's viewpoint and grasp detection performance. Specifically, when a robot already has a rough idea of where and how to grasp an object, this work investigates how to choose the best camera angle to get a more precise and confident grasp detection. This is crucial because a more reliable grasp detection can lead to a higher chance of successfully picking up an object, especially in cluttered environments.

Methods

The core of the paper's method involves creating "maps" that characterize grasp detection performance (like the density of detected grasps, true positives, false positives, and accuracy) as a function of viewpoint relative to a target grasp.

  1. Viewpoint Projection: To create these maps, viewpoints are transformed into the reference frame of the detected grasp. This allows for a consistent way to analyze performance across different grasps and objects.
  2. Performance Mapping: The authors collect data from a large dataset (BigBIRD) and for each object, they generate grasp candidates from numerous viewpoints. They then record the grasp detection score and ground truth for each candidate. These data points are aggregated into maps where the x-axis represents azimuth and the y-axis represents elevation of the viewpoint relative to the grasp.
  3. Smart Viewpoint Selection: Instead of simply maximizing accuracy, the authors propose a metric that maximizes the expected number of true positives minus the expected number of false positives. This "smart" viewpoint is chosen from the generated performance maps.
  4. Comparison Strategies: The smart viewpoint selection is compared against two baselines:
  5. Random Viewpoint: Selecting a viewpoint uniformly at random.
  6. Head-on Viewpoint: Selecting a viewpoint directly aligned with the grasp's approach vector.

Datasets

  • BigBIRD: This dataset is used extensively for offline experiments. It contains RGBD images of various objects taken from multiple viewpoints under controlled conditions. The dataset is used to generate grasp candidates and evaluate performance metrics. The authors note that BigBIRD is comprised of 125 objects, which they categorize into "box-like" and "cylinder-like" shapes, as performance maps differed between these categories.

Key Insights and Ideas

  • Viewpoint is Crucial: The right viewpoint can dramatically increase the number of detected grasps (up to 6x) and improve the confidence of the top-ranked detections. This is a fundamental finding that underpins the entire work.
  • Beyond Accuracy: Maximizing raw classification accuracy isn't always the best strategy for viewpoint selection. The authors found that accuracy is often maximized at low-density detection viewpoints, which limits the robot's choices. Their proposed metric (true positives minus false positives) balances detection quantity with quality.
  • Object Shape Matters: The optimal viewpoints differ for different object shapes (e.g., box-like vs. cylinder-like). This highlights the need for adaptive or learned viewpoint selection strategies.
  • Alignment View's Role: A "head-on" or "alignment" view, while not always optimal for detection quality, is effective at mitigating kinematic errors in the robot's arm, especially when dealing with cluttered scenes.

Key Results

  • Offline Experiments:
    • The "smart" viewpoint selection strategy detected approximately 6 times more positive grasps than random viewpoints for cylinder-like objects.
  • For box-like objects, the smart viewpoint selection improved grasp success rates by an average of 4% compared to random viewpoints in real robotic experiments.
  • The head-on alignment view improved grasp success rates by an average of 8% compared to random viewpoints.
  • Combining the smart viewpoint with the head-on alignment view resulted in the highest grasp success rates, improving them by an average of 12% over random.
  • Real Robot Experiments:
    • In a cluttered environment, using the smart viewpoint increased grasp success rates by 4% over random.
  • The alignment view alone improved success rates by 8% over random.
  • Combining both smart and alignment views yielded a 12% improvement in grasp success rate.

Limitations and Future Work

  • Limitations:
    • The paper primarily uses a single map for all grasp attempts in the robot experiments, even though object shapes can differ. This is justified by the observation that the box-like object map was also effective for cylinders.
  • The "alignment" view's significant benefit in real-world experiments is attributed partly to correcting for kinematic errors and sensor calibration issues in the specific robot setup (Baxter robot and Intel RealSense SR300).
  • Future Work:
    • The authors suggest that future work could involve learning viewpoint selection policies that are more adaptive to specific object shapes or even individual object instances.
  • Exploring more sophisticated ways to combine information from multiple views could further enhance grasp detection performance.

Summary updated

BibTeX
@inproceedings{gualtieri2017viewpoint,
  author = {Marcus Gualtieri and Robert Platt Jr.},
  title = {{Viewpoint selection for grasp detection}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2017},
  doi = {10.1109/iros.2017.8202166},
}
Figure