On Hand-Held Grippers and the Morphological Gap in Human Manipulation Demonstration

Kiran Doshi, Yijiang Huang, Stelian CorosarXiv 2023 · DOI · arXiv

Abstract

Learning from Demonstration (LfD) is a cornerstone of modern robotics, yet it faces a significant bottleneck: the "morphological gap." Human hands possess over 20 degrees of freedom (DOF) and soft tissue, while most industrial robots utilize simple two-finger parallel grippers. This disparity makes mapping human demonstrations to robotic actions computationally expensive and often inaccurate. This paper presents a novel approach to bridge this gap by using 3D-printed, hand-actuated parallel grippers. By placing the robot's interface directly in the hands of human demonstrators, we can collect scalable, robot-compatible data that captures the strategic adaptations necessary for low-DOF manipulation.

Introduction

The challenge of scaling robot learning lies in data acquisition. Robot-native data collection, such as teleoperation, is precise but slow and expensive. Conversely, human-native data (e.g., video datasets) is abundant but difficult to translate due to the morphological gap. To address this, we propose a "human-in-the-loop" hardware solution: a lightweight gripper that mimics robotic end-effectors but is controlled by human intuition. This allows us to separate strategic intelligence from mechanical constraints, identifying how humans adapt their problem-solving when limited to parallel-jaw kinematics.

Methodology

The core of our approach is a 3D-printed parallel gripper actuated by a simple pinching motion of the thumb and index finger. This hardware ensures that the demonstrations collected are inherently compatible with the kinematic constraints of common robotic platforms.

To evaluate the impact of this tool, we categorize manipulation tasks into three cases:

  • Case 1: Tasks solvable using strategies similar to bare-hand manipulation.
  • Case 2: Tasks that are impossible or fail within a time limit due to hardware constraints (e.g., requiring more than two contact points).
  • Case 3: Tasks that require a significantly modified strategy to succeed compared to bare-hand manipulation.
Figure 1: Overview of household tasks performed using hand-held parallel grippers across office, kitchen, and laundry environments.

Dataset and Task Analysis

We developed a proof-of-concept dataset encompassing diverse household activities in three settings: Office, Kitchen, and Laundry Room. Tasks ranged from simple object picking to complex multi-step processes like loading a dishwasher or folding clothes.

Our analysis revealed that many tasks traditionally thought to require high dexterity are achievable with parallel grippers if the strategy is adapted. These adaptations often involve:

  1. Iterative Re-grasping: Replacing fluid human motions with discrete steps.
  2. Environmental Support: Using tables or other surfaces as "extra fingers" to stabilize objects.
  3. Pivoting: Using one gripper as a fixed point while the other manipulates the object.
Figure 3: Comparison of strategies for picking up a book, illustrating the shift from thumb leverage to a pivoting technique.

Key Results and Insights

The study highlights that the primary limitation of current robotic manipulation is often not the hardware's mechanical capability, but the lack of demonstration data reflecting gripper-specific strategies. For instance, while "using scissors" (Case 2) proved nearly impossible due to the need for three distinct contact points, tasks like "folding socks" (Case 3) were successful once the demonstrator adopted a multi-step re-grasping approach.

We identified four specific bottlenecks of parallel grippers: reduced DOFs, limited gripping width, fewer contact points, and tactile/visual occlusion caused by the gripper's bulk. Understanding these limits allows for better hardware design and more targeted data collection.

Conclusion and Future Work

Hand-held grippers provide a scalable pathway for collecting high-quality, robot-compatible demonstration data. By leveraging human strategic intelligence to overcome mechanical constraints, we can teach robots complex tasks without the need for complex mapping algorithms. Future work will focus on integrating 6D pose tracking into the grippers to provide the precise trajectory data required for training advanced neural policies and open-sourcing the hardware design to facilitate large-scale data collection.

Reference: Doshi, K., Huang, Y., & Coros, S. (2023). On Hand-Held Grippers and the Morphological Gap in Human Manipulation Demonstration. arXiv:2311.01832v1 [cs.RO].

Summary updated

Abstract

Collecting manipulation demonstrations with robotic hardware is tedious - and thus difficult to scale. Recording data on robot hardware ensures that it is in the appropriate format for Learning from Demonstrations (LfD) methods. By contrast, humans are proficient manipulators, and recording their actions would be easy to scale, but it is challenging to use that data format with LfD methods. The question we explore is whether there is a method to collect data in a format that can be used with LfD while retaining some of the attractive features of recording human manipulation. We propose equipping humans with hand-held, hand-actuated parallel grippers and a head-mounted camera to record demonstrations of manipulation tasks. Using customised and reproducible grippers, we collect an initial dataset of common manipulation tasks. We show that there are tasks that, against our initial intuition, can be performed using parallel grippers. Qualitative insights are obtained regarding the impact of the difference in morphology on LfD by comparing the strategies used to complete tasks with human hands and grippers. Our data collection method bridges the gap between robot- and human-native manipulation demonstration. By making the design of our gripper prototype available, we hope to reduce other researchers effort to collect manipulation data.
BibTeX
@misc{doshi2023hand,
  author = {Kiran Doshi and Yijiang Huang and Stelian Coros},
  title = {{On Hand-Held Grippers and the Morphological Gap in Human Manipulation Demonstration}},
  year = {2023},
  eprint = {2311.01832},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.01832},
}
Figure