A High-Force Gripper with Embedded Multimodal Sensing for Powerful and Perception Driven Grasping

Edoardo Del Bianco, Davide Torielli, Federico Rollo, Damiano Gasperini, Arturo Laurenzi, Lorenzo Baccelliere, Luca Muratore, Marco Roveri, Nikos G. TsagarakisHumanoids 2024

TL;DR

This paper introduces a novel, modular robotic gripper that combines high grasping force (up to 110 N) with integrated multimodal sensing (camera, ToF, IMU, microphone). This allows for perception-driven grasping operations, enabling robots to better interact with and manipulate objects in complex environments.

What Problem Does This Paper Address?

Modern humanoid robots are increasingly capable of complex manipulation tasks, but their end-effectors often struggle with high-payload grasping and lack sufficient integrated perception. Existing grippers frequently rely on external sensors, which can be occluded by the robot's own arm during operation. This limits the types of objects robots can reliably grasp and manipulate, especially those requiring significant force. The paper aims to bridge this gap by developing a gripper that is both powerful and perceptually aware, directly on the end-effector.

Methods

The core of the paper is the design and validation of a modular gripper. Key aspects include:

  • High-Force Actuation: A high-power, torque-controlled Series Elastic Actuator (SEA) provides a peak pinching force of 110 N. This is a significant improvement over many existing grippers.
  • Modularity: The gripper is designed in three main modules: the main body, modular jaws, and a sensing module. This allows for easy customization of jaw shapes and materials for specific tasks, as well as straightforward upgrades to the sensing capabilities.
  • Embedded Multimodal Sensing: A compact unit integrates an eye-in-hand camera, a Time-of-Flight (ToF) distance sensor, an Inertial Measurement Unit (IMU), and an omnidirectional microphone. This provides rich sensory data directly at the point of interaction.
  • Perception-Driven Grasping Pipeline: A software pipeline leverages the multimodal sensing for tasks like object detection (using YOLOv8), alignment, and reach, all controlled via ROS. The microphone enables vocal command recognition for user interaction.
  • Novel Evaluation Metrics: The authors introduce new metrics for evaluating grasping force capacity, considering not just static payload but also the robot arm's dynamic motion and the gripper's thermal state.

Datasets

Not discussed in the paper. The paper focuses on the design and experimental validation of the gripper itself, rather than training perception models on large datasets. The YOLOv8 network is mentioned for object detection, implying it was used with its standard training data.

Key Insights and Ideas

  • Integrated Perception is Crucial for Robust Grasping: By embedding sensors directly into the gripper, the system overcomes occlusion issues and enables more responsive, perception-driven grasping.
  • Modularity Enhances Versatility: The modular design is a key enabler, allowing the gripper to be adapted for a wide range of tasks and object types, from simple parallel jaws to custom shapes.
  • Beyond Static Force: Dynamic and Thermal Considerations: Evaluating gripper performance under dynamic motion and thermal stress provides a more realistic understanding of its capabilities in real-world scenarios.
  • Human-Robot Interaction via Gripper-Mounted Mic: The integration of a microphone directly on the gripper allows for intuitive vocal commands, simplifying user interaction.

Key Results

  • High Grasping Force: The gripper can generate a peak pinching force of 110 N at a distance of 0.15 m from the joint.
  • Thermal Performance: The actuator can sustain a continuous torque of approximately 10 Nm for extended periods (over 850 seconds) with a temperature rise to 70°C, well within operational limits. Even at peak torque (16.5 Nm), it can hold the force for about 80 seconds before reaching 85°C.
  • Payload Capacity: The gripper can statically hold an 8 kg payload indefinitely and up to 10 kg for approximately 3 minutes.
  • Dynamic Grasping: The gripper successfully maintained its grasp on a 5.864 kg payload during dynamic arm movements, with acceleration limits often dictated by the robot arm's joints rather than the gripper's holding capability.
  • Perception-Driven Grasping Demonstration: Qualitative experiments demonstrated the ability to detect, align, and grasp objects using the integrated camera and ToF sensor, with vocal commands processed by the onboard microphone.

Limitations and Future Work

  • Limitations:
    • While the gripper offers high force, its single revolute joint limits its ability to conform to highly irregular shapes without specialized jaw modules.
  • The current perception-driven grasping pipeline is demonstrated qualitatively; further quantitative evaluation and robustness testing are needed.
  • Future Work:
    • Develop more sophisticated perception algorithms, particularly for monocular 3D pose estimation to address scale ambiguity.
  • Explore passive articulated or soft jaw modules to provide intrinsic adaptability and better force distribution for grasping complex shapes.
  • Further investigate the integration of multimodal sensing for more advanced autonomous grasping functionalities.

Summary updated

BibTeX
@inproceedings{bianco2024high,
  author = {Edoardo Del Bianco and Davide Torielli and Federico Rollo and Damiano Gasperini and Arturo Laurenzi and Lorenzo Baccelliere and Luca Muratore and Marco Roveri and Nikos G. Tsagarakis},
  title = {{A High-Force Gripper with Embedded Multimodal Sensing for Powerful and Perception Driven Grasping}},
  booktitle = {Humanoids},
  year = {2024},
}
Figure