Interpreting and Predicting Tactile Signals via a Physics-Based and Data-Driven Framework

Yashraj Narang, Karl Van Wyk, Arsalan Mousavian, Dieter Fox

Abstract

Tactile sensing is crucial for dexterous robotic manipulation, yet interpreting signals from complex sensors like the SynTouch BioTac remains a significant challenge. This paper introduces a hybrid framework that combines an automated experimental testbed with a 3D finite-element (FE) model to translate raw electrode impedance changes into high-resolution tactile maps. By leveraging PointNet++ architectures, the researchers demonstrate accurate estimation of both low-dimensional force vectors and high-dimensional surface deformation fields.

Introduction

While humans utilize dense tactile feedback for complex tasks, most robotic systems treat tactile sensors as simple binary switches or low-dimensional force transducers. The SynTouch BioTac sensor offers high performance but produces "cryptic" raw outputs—impedance changes across an internal electrode array—that are difficult to map to physical contact properties. This work addresses two primary gaps: accurately estimating basic features like 3D force vectors and predicting high-resolution "tactile fields" (local surface deformations) which are essential for handling fragile or irregular objects.

Overview of the framework: (A) Automated testbed, (B) 3D FE model, (C) Force data, (D) Point cloud representation.

Methodology

The proposed framework utilizes a multi-pronged approach:

  1. Automated Experimental Testbed: An ABB YuMi robot was employed to precisely indent BioTac sensors with various 3D-printed geometries. This setup ensures high-quality, repeatable ground truth data for forces and contact locations, moving beyond the limitations of manual data collection.
  2. 3D Finite Element (FE) Model: The authors developed the first 3D FE model of the BioTac sensor. This model accounts for the hyperelastic skin, internal rigid core, and fluid coupling, allowing for the simulation of "ground truth" surface deformations that are physically impossible to measure directly.
  3. Deep Learning Architecture: Using PointNet++, the researchers mapped the BioTac’s electrode array (treated as a point cloud) to both low-dimensional features (force/location) and high-dimensional fields (nodal displacements from the FE model).

The BioTac Dataset

A core contribution of this research is the release of the BioTac Dataset, which includes over 30,000 data points from 700 trajectories across three sensors. The dataset features nine unique indenter geometries, ranging from simple spheres to complex rings and textured surfaces. Notably, 70% of the trajectories focus on shear forces, which are critical for slip detection.

The nine unique 3D-printed indenter geometries used for data collection.

Key Results and Insights

The framework achieved high accuracy across several metrics:

  • FE Model Validation: The 3D FE model showed a median force error of only 0.260 N across all indenter types.
  • Contact Localization: The neural network estimated contact locations with a median error of 2.1–2.5 mm, reaching the theoretical limit imposed by the physical electrode spacing.
  • Tactile Field Prediction: The system predicted dense 3D deformation fields with a mean error of 0.22 mm, producing results visually indistinguishable from physics-based simulations.

The study also highlighted the impact of manufacturing variability; "identical" sensors produced different signal profiles, suggesting that per-sensor fine-tuning is necessary for optimal performance.

Conclusion and Future Work

This research demonstrates that validated physics models can act as powerful "labelers" for internal sensor states that cannot be directly observed. Future work will focus on addressing sensor variability through meta-learning or normalization, extending the model to dynamic interactions involving vibrations and high-speed sliding, and integrating these tactile fields into closed-loop control for delicate manipulation tasks.

Summary updated

BibTeX
@inproceedings{narang2020interpreting,
  author = {Yashraj Narang and Karl Van Wyk and Arsalan Mousavian and Dieter Fox},
  title = {{Interpreting and Predicting Tactile Signals via a Physics-Based and Data-Driven Framework}},
  booktitle = {Robotics: Science and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=C96OlMzx8o},
}
Figure