Robust Learning of Tactile Force Estimation through Robot Interaction

Balakumar Sundaralingam, Alexander Sasha Lambert, Ankur Handa, Byron Boots, Tucker Hermans, Stan Birchfield, Nathan D. Ratliff, Dieter Fox

TL;DR

This paper introduces a robust method for estimating 3D force vectors from SynTouch BioTac tactile sensors using a deep neural network with a novel voxelized input layer. By combining traditional force-torque sensor data with a clever physics-based "planar pushing" approach to generate ground truth labels, the researchers achieved a 93% improvement in force magnitude accuracy and demonstrated reliable force-feedback control for delicate robotic manipulation.

What Problem Does This Paper Address?

Tactile sensing is crucial for robots to interact safely with the world, yet translating raw sensor signals into meaningful force estimates remains a major hurdle. The SynTouch BioTac—a popular biomimetic sensor—provides complex impedance and pressure data that is notoriously difficult to model.

Existing approaches usually fall into two camps: analytic models, which are often noisy and fail during complex shear (sliding) forces, and task-specific learned models, which tend to overfit and fail when applied to new objects or motions. The challenge is creating a model that is both accurate across a wide range of forces (especially small forces under 2N) and robust enough to generalize across different robotic tasks.

Methods

The authors tackle this using a multi-pronged approach involving a new network architecture and a novel data collection strategy:

  1. Voxelized Input Layer: Instead of treating the sensor's 19 electrode readings as a simple flat list (vector), they map them into a 3D voxel grid. This preserves the spatial relationship of the electrodes on the sensor's curved surface, allowing a 3D Convolutional Neural Network (CNN) to better learn the underlying geometry of the contact.
  2. Physics-Based Force Inference: To get ground truth data for very small forces (where standard sensors are noisy), they used planar pushing. By observing how a robot pushes a box and modeling the interaction as a system of particles with Coulomb friction, they could mathematically infer the contact forces without needing a physical force-torque sensor at the contact point.
  3. Surface-Normal Regularization: They introduced a novel loss function that weights errors based on the angle between the force and the sensor's surface normal. This accounts for the hypothesis that tactile signals become less reliable as the "shear" component of the force increases.

Datasets

The researchers collected a massive dataset of 140,000 samples from three distinct sources:

  • rigid-ft (20k samples): A BioTac mounted directly to a force-torque (FT) sensor while being pressed by various objects.
  • ball-ft (20k samples): A robot finger interacting with a ball attached to an FT sensor, providing a wide variety of contact angles.
  • planar-pushing (100k samples): Data generated by a robot pushing a box, where forces were inferred via the physics model mentioned above.

Key Insights and Ideas

  • Spatial Context is King: Moving from a flat vector to a 3D voxel grid was a "lightbulb moment." It allows the network to "see" the sensor's shape, which is critical for distinguishing between a direct press and a sliding motion.
  • The "Pushing" Advantage: Standard force-torque sensors are often too noisy to capture the delicate forces (0.1N - 2.0N) involved in gentle manipulation. Using the physics of pushing provided a "cleaner" source of ground truth for these low-force regimes.
  • Cross-Source Generalization: The model trained on "mixed" data (combining all three sources) performed significantly better than models trained on just one. This suggests that diversity in how data is collected is just as important as the quantity of data.

Key Results

The proposed model outperformed the previous state-of-the-art (SOTA) by massive margins:

  • Directional Accuracy: Achieved a median angular error of 3.5 degrees (a 66% improvement over SOTA).
  • Magnitude Accuracy: Achieved a median error of 0.06 Newtons (a 93% improvement over SOTA).
  • Real-World Success: In robotic "lift and place" tasks, the model enabled the robot to successfully handle deformable objects like plastic bottles and paper cups—tasks where previous models often crushed the object or dropped it due to inaccurate force feedback.

Limitations and Future Work

While the results are impressive, the authors note a few areas for improvement:

  • Sensor Drift: Like many tactile sensors, the BioTac suffers from "drift" over time, requiring the system to be "tared" (reset) frequently.
  • Known Properties: The current planar pushing model requires knowing the object's mass and friction coefficient beforehand. Future work aims to infer these properties on the fly.
  • Complex Manipulation: The study focused on lifting and placing; the next step is applying these robust force estimates to more complex "in-hand" manipulation, like rotating an object within a robotic grasp.

Summary updated

Abstract

Current methods for estimating force from tactile sensor signals are either inaccurate analytic models or task-specific learned models. In this paper, we explore learning a robust model that maps tactile sensor signals to force. We specifically explore learning a mapping for the SynTouch BioTac sensor via neural networks. We propose a voxelized input feature layer for spatial signals and leverage information about the sensor surface to regularize the loss function. To learn a robust tactile force model that transfers across tasks, we generate ground truth data from three different sources: (1) the BioTac rigidly mounted to a force torque (FT) sensor, (2) a robot interacting with a ball rigidly attached to the same FT sensor, and (3) through force inference on a planar pushing task by formalizing the mechanics as a system of particles and optimizing over the object motion. A total of 140k samples were collected from the three sources. We achieve a median angular accuracy of 3.5 degrees in predicting force direction (66% improvement over the current state of the art) and a median magnitude accuracy of 0.06 N (93% improvement) on a test dataset. Additionally, we evaluate the learned force model in a force feedback grasp controller performing object lifting and gentle placement. Our results can be found on https: //sites.google.com/view/tactile-force.
BibTeX
@inproceedings{sundaralingam2019robust,
  author = {Balakumar Sundaralingam and Alexander Sasha Lambert and Ankur Handa and Byron Boots and Tucker Hermans and Stan Birchfield and Nathan D. Ratliff and Dieter Fox},
  title = {{Robust Learning of Tactile Force Estimation through Robot Interaction}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2019},
  url = {https://openreview.net/forum?id=vscJuFFg5rj},
}

Loading related content...

Figure