Abstract
Robotic grasp detection—identifying the optimal 2D position, orientation, and gripper aperture for picking up objects—is a fundamental challenge in autonomous manipulation. We present the Densely Supervised Grasp Detector (DSGD), a hierarchical deep learning framework that predicts robotic grasps at global, region, and pixel scales simultaneously. By ensembling these multi-level predictions and utilizing dense feature fusion, DSGD achieves state-of-the-art accuracy on standard benchmarks and demonstrates high robustness in real-world robotic trials.
Introduction
The primary hurdle in robotic grasping is generalization: a robot must reliably pick up novel objects in cluttered environments. Existing detection methods typically operate at one of three scales: global (whole image), patch-based (local crops), or pixel-level (per-pixel prediction). Each approach has weaknesses: global methods often fail on symmetric objects, while pixel-level methods can produce noisy or unstable candidates. DSGD addresses these issues by combining all three perspectives into a single, "densely supervised" architecture, allowing the system to leverage the strengths of each scale while mitigating their individual failures.
Methodology
The DSGD architecture is built upon a DenseNet backbone, which facilitates dense feature fusion to preserve high-resolution spatial details throughout the network. The framework is composed of four primary modules:
- Base Network: Extracts multi-scale features from the input image.
- Global Grasp Network (GGN): Processes the entire scene to output a single, high-confidence global grasp candidate.
- Region Grasp Network (RGN): A key innovation that identifies "salient regions" (e.g., handles or edges). It learns to segment graspable parts before predicting specific grasp parameters for those regions.
- Pixel Grasp Network (PGN): A fully convolutional network that generates dense "grasp maps," providing candidates for every pixel in the input.
Inference and Selection Logic
During inference, DSGD employs a confidence-based switching mechanism rather than simple averaging. The system prioritizes the Region-level grasp if its confidence exceeds 0.95. If this threshold is not met, it evaluates the Pixel-level prediction (threshold 0.90), finally falling back to the Global prediction if necessary. This hierarchy allows the model to "recover" from errors at any single sub-network level.
Experimental Evaluation
The authors evaluated DSGD across three distinct domains: the Cornell Grasp Dataset (benchmark), a Multi-Object Dataset (clutter), and real-world physical trials.
Key Results
- Benchmark Performance: DSGD achieved a record-breaking 97.7% accuracy on the Cornell dataset, outperforming previous state-of-the-art models.
- Robustness in Clutter: On the multi-object dataset, the system maintained 97.2% accuracy, proving its capability in complex, random orientations.
- Real-World Success: In 200 physical trials using a 7-DOF robotic arm and a Kinect sensor, the system achieved a 90% success rate on previously unseen objects.
Discussion and Insights
The success of DSGD stems from its hierarchical approach. By forcing the RGN to segment salient parts, the model anchors its predictions in physical reality (e.g., identifying a handle) rather than abstract coordinate regression. Furthermore, the use of hybrid loss functions—combining Mean Squared Error for coordinates and Cross-Entropy for angular classification—provided greater training stability.
Conclusion
DSGD demonstrates that multi-scale dense supervision is a powerful strategy for robotic grasp detection. While currently optimized for parallel-jaw grippers, future work will explore extensions to multi-fingered hands and more complex, non-planar object geometries. The "DSGD-lite" variant already achieves real-time performance (6 fps), making it suitable for closed-loop control in dynamic environments.
Summary updated