Grasping Force Control of Multi-Fingered Robotic Hands through Tactile Sensing for Object Stabilization

Zhen Deng, Yannick Jonetzko, Liwei Zhang, Jianwei Zhang2020 · DOI

TL;DR

This paper presents a novel framework for multi-fingered robotic hands to achieve stable grasping of unknown objects by integrating tactile sensing with force control. The system uses a deep neural network to simultaneously detect slip and object material from tactile data, and a Gaussian Mixture Model to estimate contact force and location, enabling adaptive grasping force adjustments.

What Problem Does This Paper Address?

Robots performing manipulation tasks in everyday environments need to stably grasp objects, even when faced with external disturbances or unknown object properties. Humans excel at this by using tactile feedback to adjust their grip force and react to instabilities. However, applying this to robotic hands, especially multi-fingered ones, is challenging. Existing approaches often treat slip detection and material identification separately, or rely on manually engineered features. This paper aims to bridge this gap by developing an integrated system that uses tactile sensing to dynamically adjust grasping force for stable object manipulation without prior knowledge of the object's shape or material.

Methods

The core of the proposed system is a three-component framework:

  • Online Detection Module: This module uses a Deep Neural Network (DNN), specifically an LSTM-based encoder-decoder architecture, to process tactile sequences. It simultaneously detects two critical pieces of information: the contact event (non-contact, contact, or slip) and the object's material (paper, foam, plastic, metal). The LSTM captures the temporal dynamics of the tactile data, and the encoder-decoder learns a low-dimensional latent representation that is then fed into separate slip and material classifiers. This joint detection approach is a key novelty.
  • Force Estimation Method: This component leverages the spatial properties of tactile data. It first constructs a "tactile image" from the sensor's electrode readings. Then, it uses a Gaussian Mixture Model (GMM) to cluster the tactile image, identifying the contact region. Finally, it estimates the contact force and location based on the values of the electrodes within this identified contact region. This method aims to reduce uncertainties by focusing only on contacted electrodes.
  • Grasping Force Controller: This is a force feedback controller that drives the robotic hand's joints. It uses the information from the detection module and force estimation to dynamically update the desired grasping force. The desired force is initially set based on the detected material (e.g., lower force for foam, higher for metal) and then adjusted in real-time. If slip is detected, the grasping force is increased; otherwise, it's maintained. A PID controller then ensures the robotic hand tracks this desired force.

Datasets

The paper introduces a new tactile dataset specifically for simultaneous slip and material detection. This dataset includes tactile data, ground truth contact events (non-contact, contact, slip), and object material labels (paper, foam, plastic, metal). Twelve household objects were used, categorized into four material groups. Tactile data was collected using a Shadow Dexterous Hand with BioTac sensors at 100 Hz. The dataset was split into 90% for training and 10% for testing.

Key Insights and Ideas

  • Joint Slip and Material Detection: The paper demonstrates that detecting slip and material simultaneously from tactile data is more effective and efficient than separate approaches. This allows for a more informed and adaptive grasping strategy.
  • LSTM for Temporal Tactile Features: Utilizing LSTMs effectively captures the temporal dynamics of tactile sensor readings, which are crucial for understanding events like slippage.
  • Spatial Tactile Data for Force Estimation: Focusing on the spatial distribution of pressure within the contact region, rather than the entire sensor surface, leads to more robust force estimation by mitigating noise from non-contact areas.
  • Material-Informed Initial Grasping Force: The idea that different materials require different initial grasping forces (e.g., foam vs. metal) is a practical insight derived from human grasping behavior.
  • Adaptive Grasping Force: The framework's ability to dynamically adjust grasping force based on real-time tactile feedback (slip and material) is key to stabilizing unknown objects.

Key Results

  • Online Detection Module Performance: The slip detection achieved approximately 98% accuracy, and material detection achieved about 95% accuracy on the proposed dataset. Confusion matrices showed good separation between classes, with some minor confusion between plastic and paper materials due to similar physical properties.
  • Parameter Sensitivity: Increasing the window size for tactile sequence sampling generally improved detection accuracy, while sampling rate had a less consistent effect. The authors selected a window size of $l=32$ and a sampling rate of $f=33.3$ Hz for real-world experiments to balance accuracy and detection delay.
  • Real-World Grasping Experiment: The proposed framework demonstrated successful object stabilization in real-world experiments with a Shadow Dexterous Hand. The grasp success rate for eight different objects was 95% (76/80), with failures primarily occurring with a metal mug. This indicates the system's effectiveness in handling unknown objects and external disturbances.

Limitations and Future Work

  • Force Calibration: The current force estimation method provides a relative measure of contact force, not an absolute real force value. Future work plans to use a force/torque sensor for calibration to obtain ground truth data.
  • Sensor Specificity: The tactile sensing methods are developed based on BioTac sensors. While the general framework can be adapted, direct transfer to other sensor types might require modifications, particularly in the tactile image generation step.
  • Material Classification Granularity: The current material classification is limited to four broad categories. Future work could explore more refined material classification and potentially detect objects made of multiple materials.
  • Friction Detection: Object friction is an important factor for grasping, but it was not addressed. Detecting friction would require specialized measurement setups and is a challenging area for future research.
  • Fusion with Visual Data: Tactile sensors provide local information, while visual sensors offer global context. The authors suggest fusing visual and tactile modalities for more robust robotic perception.
  • Contact Point Adjustment: While the current work focuses on adjusting grasping force, future research could also explore adjusting the contact point of the fingers for even more stable grasps, which would require integrating global object and environment information.

Summary updated

BibTeX
@misc{deng2020grasping,
  author = {Zhen Deng and Yannick Jonetzko and Liwei Zhang and Jianwei Zhang},
  title = {{Grasping Force Control of Multi-Fingered Robotic Hands through Tactile Sensing for Object Stabilization}},
  year = {2020},
  doi = {10.3390/s20041050},
}
Figure