Skeleton-Guided Geometry-Aware Auto-Labeling for Robotic Grasping: From 4-DoF Learning to 6-DoF Execution

Ali Sabzejou, Farbod Azimmohseni, Mehdi Tale Masouleh, Ahmad KalhorarXiv 2026 · DOI · arXiv

TL;DR

This paper introduces a geometry-aware framework that uses "straight skeletons" to automatically label robotic grasp points, eliminating the need for costly manual annotation. By integrating these geometric priors into a novel neural network (SkelGG-CNN) and extending the method to 6-DoF point clouds, the authors achieve human-level grasping precision and high real-world success rates even with limited training data.

What Problem Does This Paper Address?

Deep learning has revolutionized robotic grasping, but it has a "data hunger" problem. Training robust models typically requires massive datasets of objects labeled with successful grasp poses. Creating these datasets manually is labor-intensive, expensive, and prone to human inconsistency.

While analytical methods (based on physics and geometry) can find grasps without manual labels, they are often too computationally heavy for real-time use or struggle with the noisy, partial data seen by real-world sensors. The authors aim to bridge this gap by using geometry to "teach" the neural network where to look, creating a pipeline that is both automated and highly efficient.

Methods

The core innovation is the use of Straight Skeletons—a geometric representation that acts like a "spine" for 2D shapes.

  1. Auto-Labeling Pipeline: The system extracts the skeleton of an object's contour. It then uses a pairing algorithm to find "antipodal" points (points on opposite sides of the object that allow for a stable squeeze) centered around the skeleton's nodes. This ensures grasps are physically sensible and located in high-potential regions.
  2. SkelGG-CNN: The authors propose a new architecture based on Generative Grasp CNNs. Unlike standard models that predict quality, angle, and width, SkelGG-CNN adds a fourth "Skeleton Heatmap" head. This heatmap guides the model to refine grasp centers toward the object's geometric medial axis, significantly improving stability.
  3. Generalized 6-DoF Execution: To handle 3D space, they developed a "quasi-3D" approach. They slice a 3D point cloud into horizontal 2D layers, apply skeletonization to each slice, and then link them back together to form a volumetric skeleton graph.
Figure 2. Overview of the proposed method. The pipeline begins with the simulation environment, where RGB-D images are captured from a set of 3D meshes sourced from the Dex-Net dataset. The subsequent stages show the details process of the straight skeleton-based grasp pose estimation algorithm. After this process, the generated grasp poses are used to create the SkelGG-CNN model labels. The skeleton heatmap is generated during the polygon offsetting phase of the algorithm, adding geometric features to the model.

Datasets

The framework was validated across a wide range of standard benchmarks:

  • Jacquard V1 & V2: Large-scale synthetic datasets for 4-DoF grasping.
  • Cornell Grasping Dataset: A classic benchmark with human-annotated labels.
  • Dex-Net: Used for simulation-based end-to-end evaluation.
  • GraspNet-1Billion: Used to test the 6-DoF generalized auto-labeling in cluttered scenes.
  • EGAD!: A dataset specifically designed to test generalization on complex, diverse geometries.

Key Insights and Ideas

  • Skeletons as Grasp Priors: The straight skeleton naturally decomposes complex shapes into local regions that are ideal for two-finger grasping. By focusing the search space on the skeleton, the algorithm avoids the "random sampling" inefficiency of many other methods.
  • Geometry-Aware Refinement: Standard generative models often predict high-quality grasps near object edges, which can lead to slips. The skeleton heatmap forces the model to prioritize the "center of mass" of a local geometry, leading to much more robust grasps.
  • Data Efficiency: Because the skeleton provides such a strong geometric hint, the SkelGG-CNN can achieve high accuracy even when trained on significantly smaller datasets (e.g., ~2,600 images from Dex-Net) compared to traditional methods.
Figure 3. Comparison of SkelGG-CNN output heatmaps with the pretrained Gr-ConvNetV2 Kumra et al. (2022) model. The first three columns display the quality and angle heatmaps for both models. The fourth column represents the width heatmap for Gr-ConvNet, whereas SkelGG-CNN uses a skeleton heatmap instead, enabling both grasp width estimation and grasp refinement.

Key Results

  • Benchmark Accuracy: SkelGG-CNN achieved 95.42% on Jacquard V1, 96.55% on Jacquard V2, and 97.1% on Cornell.
  • 6-DoF Performance: On the challenging GraspNet "novel" split, the auto-labeling algorithm achieved an Average Precision (AP) of 18.33, which is competitive with state-of-the-art model-based approaches that require extensive training.
  • Real-World Success: Using a Delta parallel robot, the system achieved a 97.4% success rate on single household objects and 95.0% in cluttered scenes. Crucially, this was achieved without any domain adaptation or real-world fine-tuning.

Limitations and Future Work

While the "quasi-3D" slicing method is efficient, the authors note it can occasionally neglect complex volumetric patterns that don't align with horizontal slices. Future work aims to develop algorithms that explicitly capture 3D topological structures without the slicing heuristic. Additionally, the current framework is optimized for two-finger grippers; extending this to multi-fingered hands and integrating tactile feedback are logical next steps to improve grasp stability in even more complex manipulation tasks.

Summary updated

Abstract

Robotic grasping relies on large manually annotated datasets, which are costly to create. We introduce a straight-skeleton-based algorithm for generating two-finger grasp keypoints and identifying regions suitable for stable, feasible grasps. Our automated pipeline produces consistent grasp annotations without human intervention. We also propose the Skeleton-based Generative Grasp Convolutional Neural Network (SkelGG-CNN), a 4-DoF architecture that learns geometric grasp features from skeleton-based guidance. To extend auto-labeling to 6-DoF grasping, we develop a generalized quasi-3D skeletonization algorithm that operates directly on point clouds. Using automatically generated 4-DoF labels, SkelGG-CNN achieves accuracies of 95.42%, 96.55%, and 97.1% on Jacquard V1, Jacquard V2, and Cornell, respectively. On the GraspNet novel split, 6-DoF auto-labeling achieves an average precision of 18.33, comparable to state-of-the-art model-based approaches. In simulation, the 6-DoF method achieves success rates of 98.4% on Dex-Net and 89.1% on EGAD!. Real-world experiments with a Delta parallel robot show success rates of 97.4% on single household objects and 95.0% in cluttered scenes, outperforming state-of-the-art methods without domain adaptation. The 4-DoF pipeline adapts across environments and produces annotations approaching human-level precision, enabling efficient training of lightweight models on small datasets. The generalized 6-DoF method provides scalable, geometry-aware annotations for cluttered 3D scenes.
BibTeX
@misc{sabzejou2026skeleton,
  author = {Ali Sabzejou and Farbod Azimmohseni and Mehdi Tale Masouleh and Ahmad Kalhor},
  title = {{Skeleton-Guided Geometry-Aware Auto-Labeling for Robotic Grasping: From 4-DoF Learning to 6-DoF Execution}},
  year = {2026},
  eprint = {2610.38443},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38443},
}
Figure