Crossing the Reality Gap in Tactile-Based Learning

Ya-Yen Tsai, Bidan Huang, Yu Zheng, Lei Han, Wang Wei Lee, Edward Johns

TL;DR

This paper tackles the challenge of training robotic manipulation policies using tactile feedback by developing a novel, efficient tactile sensor simulation. They propose a method to bridge the "reality gap" between simulation and the real world by identifying tactile sensor parameters as probabilistic distributions for domain randomization and using encoders to align tactile signals in a latent space, enabling successful sim-to-real transfer for grasping tasks.

What Problem Does This Paper Address?

Robotic manipulation, especially tasks requiring fine motor control and object interaction, heavily relies on sensory feedback. Tactile sensing is crucial for understanding contact forces, object properties, and navigating situations where vision is occluded. However, training robust manipulation policies with tactile data is hindered by two main issues: 1) the difficulty and cost of collecting large amounts of real-world tactile data, and 2) the significant discrepancy, or "reality gap," between tactile sensor data from simulations and actual physical sensors. Existing tactile simulations often focus on high fidelity but are too computationally expensive for large-scale policy training, and they don't adequately address the sim-to-real transfer problem. This paper aims to create an efficient tactile simulation that can be used for policy training and effectively bridge the reality gap for tactile-based manipulation.

Methods

The authors propose a two-pronged approach to address the sim-to-real gap in tactile-based learning:

  1. Mass-Spring-Damper Tactile Model with Probabilistic Parameter Identification:
    • Each individual sensing unit (taxel) of a pressure sensor array is modeled as a mass-spring-damper system. This is a simplification compared to more complex Finite Element Methods (FEM) but offers significant computational advantages for faster simulation.
  2. Crucially, instead of identifying single deterministic values for the mass-spring-damper parameters (stiffness, damping, etc.), they are identified as probabilistic distributions. This is achieved by performing the same indentation experiments in both reality and simulation and using an optimization strategy (Covariance Matrix Adaptation Evolution Strategy - CMAES) to find parameter ranges that best match the real sensor's responses.
  3. These probabilistic distributions are then used for domain randomization during policy training. By sampling parameters from these distributions, the simulation environment becomes more varied, making the trained policy more robust to variations and uncertainties in the real world.
  4. Latent Space Alignment with Tactile Encoders:
    • Even with calibrated parameters and domain randomization, raw simulated tactile signals may not perfectly match real ones. To further bridge this gap, they introduce a tactile embedding strategy.
  5. Two encoders are used: a simulation encoder (trained to encode and reconstruct simulated tactile signals) and a real encoder (trained to encode real tactile signals).
  6. These encoders are trained to map both simulated and real tactile signals into a common latent space. This means that critical tactile features, regardless of their origin (simulated or real), are represented similarly in this latent space. The goal is to minimize the reality gap in this compressed, abstract representation.

Datasets

The paper does not explicitly mention or introduce new large-scale datasets. Instead, it relies on data collected from:

  • Calibration experiments: Indentation experiments performed on both the real tactile sensor and its simulation counterpart. This data is used to identify the probabilistic parameters of the mass-spring-damper model.
  • Exploratory training: Data collected from initial, less structured training runs in simulation to train the simulation encoder.
  • Grasping experiments: Objects used for training and testing grasping policies in both simulation and reality. The paper lists specific objects used for training (four objects) and testing (six unseen objects).

Key Insights and Ideas

  • Probabilistic parameter identification is key for domain randomization: Treating tactile sensor parameters as distributions rather than fixed values is a powerful way to create diverse simulation environments that improve policy robustness for sim-to-real transfer. This moves beyond simple randomizing of object properties.
  • Latent space alignment is a crucial second step: While domain randomization helps, it doesn't eliminate all discrepancies. Projecting tactile signals into a shared latent space using encoders effectively "aligns" the representations, making them more comparable and reducing the reality gap at a higher level of abstraction.
  • Efficiency matters for tactile learning: The mass-spring-damper model, while simplified, offers a good trade-off between fidelity and computational speed, making it suitable for the large-scale data collection required for reinforcement learning.
  • Tactile feedback is essential for precision grasping: The experiments demonstrate that tactile feedback significantly improves grasping success rates, especially for smaller or more challenging objects, where visual information alone might be insufficient.

Key Results

  • Improved Tactile Model Accuracy: The calibrated tactile sensor model in simulation closely follows the real sensor's responses, significantly outperforming the uncalibrated model.
  • Successful Sim-to-Real Transfer: A grasping policy trained using the proposed tactile simulation and sim-to-real strategy achieved high success rates on unseen objects in the real world without further fine-tuning.
  • Superior Performance in Grasping: The proposed approach consistently outperformed several baselines in grasping tasks, particularly for smaller and more challenging objects, demonstrating the effectiveness of tactile feedback and the proposed sim-to-real strategy. For instance, the proposed approach showed better performance for smaller objects compared to policies without tactile feedback.
  • Encoder Effectiveness: The latent space alignment with encoders was shown to be effective in further reducing the sim-to-real gap, as evidenced by the improved performance of the proposed approach compared to baselines that only used domain randomization or simpler encoding strategies.

Limitations and Future Work

  • Limitations of the Tactile Model: The paper acknowledges that the mass-spring-damper model is a simplification and neglects force propagation between adjacent taxels. While this is justified by the stiffness of the rubber pad and the desire for speed, it might limit fidelity in certain scenarios. The model also doesn't explicitly account for time-variant responses, material properties, or temperature effects.
  • Future Work: The authors suggest that their framework is generalizable to other robot learning methods and tactile sensor arrays. Future work could explore more sophisticated tactile models if higher fidelity is required, or investigate how to further improve the efficiency and robustness of the sim-to-real transfer process. They also hint at the potential for this approach to be applied to more complex manipulation tasks beyond grasping.

Summary updated

Abstract

Tactile sensors are believed to be essential in robotic manipulation, and prior works often rely on experts to reason the sensor feedback and design a controller. With the recent advancement in data-driven approaches, complicated manipulation can be realised, but an accurate and efficient tactile simulation is necessary for policy training. To this end, we present an approach to model a commonly used pressure sensor array in simulation and to train a tactile-based manipulation policy with sim-to-real transfer in mind. Each taxel in our model is represented as a mass-spring-damper system, in which the parameters are iteratively identified as plausible ranges. This allows a policy to be trained with domain randomisation which improves its robustness to different environments. Then, we introduce encoders to further align the critical tactile features in a latent space. Finally, our experiments answer questions on tactile-based manipulation, tactile modelling and sim-to-real performance.
BibTeX
@misc{tsai2023crossing,
  author = {Ya-Yen Tsai and Bidan Huang and Yu Zheng and Lei Han and Wang Wei Lee and Edward Johns},
  title = {{Crossing the Reality Gap in Tactile-Based Learning}},
  year = {2023},
  eprint = {2305.09870},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.09870},
}
Figure