GraspNet: An Efficient Convolutional Neural Network for Real-time Grasp Detection for Low-powered Devices

Umar Asif, Jianbin Tang, Stefan HarrerIJCAI 2018 · DOI

TL;DR

This paper introduces GraspNet, a novel, efficient convolutional neural network designed for real-time grasp detection on low-power devices. By combining novel "Dilated Dense Fire" modules with an encoder-decoder architecture, GraspNet achieves competitive accuracy with significantly reduced computational requirements, making it suitable for embedded robotic systems.

What Problem Does This Paper Address?

Grasp detection, the task of identifying suitable regions on an object for a robot to grasp, is crucial for robotic manipulation. While deep learning has significantly improved grasp detection accuracy, these models often come with a substantial computational cost, requiring large memory footprints and high processing power. This makes them impractical for deployment on resource-constrained embedded systems, such as those found in many mobile robots or edge devices. Existing approaches either sacrifice accuracy for speed or demand extensive training data and computational resources, leaving a gap for efficient, accurate, and deployable grasp detection solutions.

Methods

The core of GraspNet is its novel Dilated Dense Fire (DDF) module. This module builds upon the "Fire module" concept (used in SqueezeNet) by incorporating:

  • Squeeze and Dilated Convolutions: These allow the network to capture multi-scale contextual information efficiently.
  • Dense Residual Connections: Inspired by DenseNets, these connections allow features from earlier layers to be reused by later layers, promoting deeper supervision and reducing overfitting.
  • Integrated Compression Hyper-parameters ($\alpha$ and $\delta$): These parameters allow for explicit control over the network's depth and spatial resolution, enabling a trade-off between model size, speed, and accuracy.

GraspNet employs an encoder-decoder architecture:

  • The encoder progressively downsamples the input RGB-D image using DDF modules to extract features.
  • The decoder upsamples these features using deconvolution layers to produce a pixel-wise grasp affordance segmentation map. Skip connections from the encoder are fused into the decoder to preserve fine-grained information.

The task is framed as pixel-wise grasp affordance segmentation, where each pixel is classified as either a graspable region or background.

Datasets

  • GraspSeg Dataset: A new dataset introduced by the authors, containing 33,188 RGB-D images of 42 object instances for training and 6896 images for testing. It features pixel-wise annotations for grasp affordances and object segmentations. The dataset is designed to be challenging, with test images containing multiple objects, partial occlusions, and background clutter.
  • Cornell Grasp Dataset: A widely used benchmark dataset containing 885 RGB-D images of 240 objects, with ground truth in the form of grasp rectangles.

Key Insights and Ideas

  • Efficient Feature Fusion is Key: The combination of dense residual connections and element-wise summation/concatenation in the DDF modules effectively allows for rich feature reuse and implicit deep supervision, leading to better discriminative features without excessive parameters.
  • Controlled Compression for Efficiency: The introduction of hyper-parameters $\alpha$ and $\delta$ provides a principled way to tune the network's complexity, allowing researchers to find optimal trade-offs for specific deployment scenarios (e.g., prioritizing speed over absolute accuracy for very low-power devices).
  • Framing Grasp Detection as Segmentation: Treating grasp detection as a pixel-wise segmentation task (grasp affordance segmentation) allows for more detailed and precise localization of graspable regions compared to traditional bounding box regression.

Key Results

  • High Accuracy on GraspSeg: GraspNet achieved an average grasp accuracy of 87.3% and affordance segmentation accuracy of 73.5% on the GraspSeg dataset, outperforming state-of-the-art computation-efficient models like SqueezeNet variants, MobileNet, and ENet.
  • Real-time Performance on Embedded Hardware: GraspNet achieved an inference time of 19 ms on an Nvidia Jetson TX1, making it suitable for real-time applications on low-powered devices. This is significantly faster than many comparable models.
  • Competitive on Cornell Grasp Dataset: GraspNet achieved a grasp accuracy of 90.6% (object-wise) and 90.2% (image-wise) on the Cornell Grasp Dataset, with an inference speed of only 24 ms.
  • Model Size Reduction: GraspNet's model size is significantly smaller than many other CNNs, with a base configuration of 7.2 MB, making it ideal for embedded systems.

Limitations and Future Work

  • Failure Cases: The paper acknowledges that GraspNet can still fail to predict the correct orientation of grasps for some objects (as shown in Figure 4).
  • Future Work:
    • Extend GraspNet for simultaneous object recognition and grasp detection to further reduce overall system latency.
  • Implement CRF-based post-processing to improve grasp affordance segmentation.
  • Further explore the design space with more extensive parameter searches to optimize the trade-off between accuracy, model size, and inference speed.

Summary updated

BibTeX
@inproceedings{asif2018graspnet,
  author = {Umar Asif and Jianbin Tang and Stefan Harrer},
  title = {{GraspNet: An Efficient Convolutional Neural Network for Real-time Grasp Detection for Low-powered Devices}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/677},
}
Figure