Abstract: In this paper, we present a real-time approach to predict multiple grasping poses for a parallel-plate robotic gripper using RGB images. A model with oriented anchor box mechanism is proposed and a new matching strategy is used during the training process. An end-to-end fully convolutional neural network is employed in our work. The network consists of two parts: the feature extractor and multi-grasp predictor. The feature extractor is a deep convolutional neural network. The multi-grasp predictor regresses grasp rectangles from predefined oriented rectangles, called oriented anchor boxes, and classifies the rectangles into graspable and ungraspable. On the standard Cornell Grasp Dataset, our model achieves an accuracy of 97.74% and 96.61% on image-wise split and object-wise split respectively, and outperforms the latest state-of-the-art approach by 1.74% on image-wise split and 0.51% on object-wise split.
Introduction
For a robot to pick up an object using a parallel-plate gripper, it needs to determine a precise 5D grasp configuration: location $(x, y)$, dimensions (width $w$, height $h$), and orientation $(\theta)$. Previous deep learning approaches often treated grasp detection like standard object detection, using axis-aligned anchor boxes. This meant the network had to "learn" the rotation as a separate classification or regression task decoupled from the box's geometry. This decoupling is inefficient because a gripper's feasible orientation is usually tied closely to its local geometry. Furthermore, many existing models were slow or struggled to predict multiple valid grasp points for a single object.
Methodology
The authors propose an end-to-end Fully Convolutional Grasp Detection Network. The architecture consists of two main components: a feature extractor and a multi-grasp predictor.
- Feature Extractor: A deep residual network (ResNet-50 or ResNet-101) that processes the input RGB image into high-level feature maps.
- Multi-Grasp Predictor: Instead of fully connected layers, this uses convolutional layers to predict "graspability" scores and bounding box offsets for every grid cell in the image.
Oriented Anchor Boxes
The core innovation is the Oriented Anchor Box mechanism. Standard anchor boxes vary in scale and aspect ratio but are always horizontal. In this work, the authors tile the image with anchors that have predefined default rotation angles (e.g., $15^\circ, 45^\circ, 75^\circ, -15^\circ, -45^\circ, -75^\circ$).
By starting with an anchor that is already close to the correct orientation, the network only needs to regress a small angular offset. This makes the optimization much easier and more accurate than trying to predict a $0^\circ$ to $180^\circ$ angle from a horizontal box.
Efficient Matching Strategy
During training, the authors use a simplified matching strategy. Instead of calculating complex Intersection over Union (IoU) for every anchor, they assign a positive label if the anchor center and ground-truth center fall in the same grid cell and the orientation difference is within a specific threshold ($90^\circ/k$). This approach is computationally cheaper and focuses the network on the most relevant geometric features.
Evaluation and Results
The model was evaluated on the Cornell Grasp Dataset, which contains 885 images of 240 different objects. The authors used two standard evaluation splits: Image-wise split (testing generalization to new positions) and Object-wise split (testing generalization to entirely new objects).
Key Results
- Accuracy: Achieved 97.74% on the image-wise split and 96.61% on the object-wise split.
- Performance: Outperformed the previous state-of-the-art by 1.74% and 0.51% respectively.
- Speed: The ResNet-50 version runs at 9.89 FPS, making it viable for real-time robotic control loops.
Discussion and Insights
Treating orientation as an inherent property of the anchor box significantly improves regression stability. The authors found that the quality of anchors matters more than quantity; they achieved superior results using fewer total predictions (600 vs. 1764 in previous work). Additionally, while ResNet-50 and ResNet-101 performed similarly on seen objects, the deeper ResNet-101 was notably better at the object-wise split, indicating that depth is crucial for "zero-shot" generalization to novel objects.
Future work will focus on improving "grasp relationship parsing" for cluttered scenes and optimizing the model through network pruning to increase frame rates for lower-power embedded systems.
Summary updated