Fully Convolutional Grasp Detection Network with Oriented Anchor Box

Xinwen Zhou, Xuguang Lan, Hanbo Zhang, Zhiqiang Tian, Yang Zhang, Nanning Zheng

TL;DR

This paper introduces a real-time, fully convolutional neural network (FCN) that detects multiple robotic grasp poses from a single RGB image. By incorporating a novel Oriented Anchor Box mechanism—where reference rectangles are predefined with specific rotation angles—the model achieves state-of-the-art accuracy (97.74%) on the Cornell Grasp Dataset while maintaining high-speed performance suitable for real-world robotics.

What Problem Does This Paper Address?

For a robot to pick up an object using a parallel-plate gripper, it needs to know more than just "where" the object is; it needs a precise 5D grasp configuration: location $(x, y)$, dimensions (width, height), and orientation $(\theta)$.

Previous deep learning approaches often treated grasp detection like standard object detection, using axis-aligned anchor boxes. This meant the network had to "learn" the rotation as a separate classification or regression task decoupled from the box's geometry. This decoupling is inefficient because a gripper's feasible orientation is usually tied closely to its local geometry. Furthermore, many existing models were slow or struggled to predict multiple valid grasp points for a single object.

Methods

The authors propose an end-to-end Fully Convolutional Grasp Detection Network. The architecture consists of two main components:

  1. Feature Extractor: A deep residual network (ResNet-50 or ResNet-101) that processes the input RGB image into high-level feature maps.
  2. Multi-Grasp Predictor: Instead of fully connected layers, this uses convolutional layers to predict "graspability" scores and bounding box offsets for every grid cell in the image.

The "Aha!" Moment: Oriented Anchor Boxes

The core innovation is the Oriented Anchor Box mechanism. Standard anchor boxes vary in scale and aspect ratio but are always horizontal. Here, the authors tile the image with anchors that have predefined default rotation angles (e.g., $15^\circ, 45^\circ, 75^\circ, -15^\circ, -45^\circ, -75^\circ$).

By starting with an anchor that is already close to the correct orientation, the network only needs to regress a small angular offset. This makes the optimization much easier and more accurate than trying to predict a $0^\circ$ to $180^\circ$ angle from a horizontal box.

Efficient Matching Strategy

During training, the authors use a simplified matching strategy. Instead of calculating complex Intersection over Union (IoU) for every anchor, they assign a positive label if:

  • The anchor center and ground-truth center fall in the same grid cell.
  • The orientation difference is within a specific threshold ($90^\circ/k$). This approach is computationally cheaper and focuses the network on the most relevant geometric features.

Datasets

The model was evaluated on the Cornell Grasp Dataset, which contains 885 images of 240 different objects with manually labeled "good" grasp rectangles. The authors used two standard evaluation splits:

  • Image-wise split: Tests the ability to recognize grasps on previously seen objects in new positions/orientations.
  • Object-wise split: Tests the ability to generalize to entirely new, unseen objects.

To prevent overfitting on this relatively small dataset, they employed extensive data augmentation, including random cropping, rotation, flipping, and color jittering.

Key Insights and Ideas

  • Orientation as Geometry: Treating orientation as an inherent property of the anchor box (rather than a semantic label) significantly improves regression stability.
  • Density vs. Accuracy: The authors found that they could achieve better results than previous models using fewer total predictions (600 vs. 1764), suggesting that the quality of the anchors matters more than the quantity.
  • Depth and Generalization: While ResNet-50 and ResNet-101 performed similarly on the image-wise split, the deeper ResNet-101 was notably better at the object-wise split. This indicates that deeper networks are crucial for the "zero-shot" generalization required to grasp novel objects.
  • Robustness to Occlusion: Qualitative results showed the model could successfully predict grasps even in complex scenes with overlapping objects and transparent materials (like glasses), despite these not being well-represented in the training set.

Key Results

  • Accuracy: Achieved 97.74% on the image-wise split and 96.61% on the object-wise split.
  • Performance: Outperformed the previous state-of-the-art (Chu et al.) by 1.74% and 0.51% respectively.
  • Speed: The ResNet-50 version runs at 9.89 FPS, and the ResNet-101 version runs at 8.51 FPS on an NVIDIA TITAN-X, making it viable for real-time robotic control loops.

Limitations and Future Work

  • Dataset Size: The Cornell Grasp Dataset is small by modern deep learning standards. The authors suggest that a larger dataset with more detailed annotations would further improve performance.
  • Multi-Object Parsing: While the model handles some complexity, the authors aim to improve "grasp relationship parsing"—essentially understanding which object to grab first when they are piled on top of each other.
  • Model Optimization: Future work will look into network pruning to remove unnecessary channels, potentially pushing the frame rate even higher for lower-power embedded systems on mobile robots.

Summary updated

Abstract

In this paper, we present a real-time approach to predict multiple grasping poses for a parallel-plate robotic gripper using RGB images. A model with oriented anchor box mechanism is proposed and a new matching strategy is used during the training process. An end-to-end fully convolutional neural network is employed in our work. The network consists of two parts: the feature extractor and multi-grasp predictor. The feature extractor is a deep convolutional neural network. The multi-grasp predictor regresses grasp rectangles from predefined oriented rectangles, called oriented anchor boxes, and classifies the rectangles into graspable and ungraspable. On the standard Cornell Grasp Dataset, our model achieves an accuracy of 97.74% and 96.61% on image-wise split and object-wise split respectively, and outperforms the latest state-of-the-art approach by 1.74% on image-wise split and 0.51% on object-wise split.
BibTeX
@inproceedings{zhou2018fully,
  author = {Xinwen Zhou and Xuguang Lan and Hanbo Zhang and Zhiqiang Tian and Yang Zhang and Nanning Zheng},
  title = {{Fully Convolutional Grasp Detection Network with Oriented Anchor Box}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  doi = {10.1109/iros.2018.8594116},
}

Loading related content...

Figure