Unsupervised preprocessing for Tactile Data

Maximilian Karl, Justin Bayer, Patrick van der SmagtarXiv 2016 · OpenReview

Abstract

Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record ground truth reference data. These compact representations can either be used directly in a reinforcement learning based controller or can be used to calibrate the tactile sensor to physical quantities with only a few datapoints. We show the quality of our latent representation by predicting important features and with a simple control task.

Introduction: The Challenge of Tactile Sensing

In modern robotics, achieving human-like dexterity requires sophisticated tactile feedback. While advanced sensors like the BioTac provide rich data streams, their output is notoriously difficult to utilize in real-time control. These sensors are typically high-dimensional, non-linear, and exhibit high degrees of correlation between different physical modalities.

Internal structure of the BioTac sensor, showing the complex coupling of fluid, elastomeric skin, and various sensing elements.

The primary obstacles to widespread tactile integration include:

  1. Complexity: Soft, deformable sensors mean a single contact point stimulates a wide array of sensing elements (taxels).
  2. Coupling: Physical properties, such as surface texture and applied force, are often muddled within the raw signal.
  3. Calibration Requirements: Traditional supervised methods require exhaustive manual calibration for every individual sensor, a process that must be repeated if the sensor is replaced or the task environment changes.

This research explores whether machine learning can "untangle" these signals unsupervised, discovering the underlying structure of touch without human-labeled ground truth.

Methodology: Unsupervised Feature Extraction via VAEs

The core of the proposed approach is the use of a Variational Auto-Encoder (VAE) for unsupervised preprocessing. Unlike supervised models that require explicit labels (e.g., "this is 2 Newtons of force"), the VAE learns by attempting to compress and then reconstruct the raw sensor data.

The Architecture

The VAE consists of an encoder that compresses raw taxel data into a low-dimensional "latent space" and a decoder that reconstructs the original signal from this compact representation. By forcing the data through this bottleneck, the model naturally factorizes the information, identifying independent components of the signal—such as force direction or material hardness—and mapping them to specific latent dimensions.

Linearization

A significant advantage of this method is linearization. While raw sensor responses are highly non-linear, the resulting latent variables often exhibit a nearly linear relationship with physical quantities. This makes the data significantly more accessible for simple, efficient controllers.

Experimental Setup and Datasets

To evaluate the method, the researchers utilized a custom 3-DoF robot setup equipped with a gimbal platform to collect data from two distinct sensor types: the resistive BioTac and the capacitive iCub sensor.

The automated data collection platform (a) and (b) used to generate high-dimensional tactile datasets across varying forces and orientations.

Three primary datasets were generated:

  • Dataset A (Surface Angle): Varied roll and pitch angles with forces ranging from 0 to 5 N.
  • Dataset B (Shore Hardness): Utilized silicone samples of varying hardness levels measured at multiple angles.
  • Dataset C (Curvature): Involved five discrete spherical curvatures and one flat surface.

Results and Discussion

Emergent Physics

The VAE successfully "discovered" physical properties without explicit instruction. In the latent space, specific nodes correlated almost perfectly with force or angle, despite the model never being exposed to labeled physical units during training. This suggests the VAE learns the fundamental structure of the interaction rather than hardware-specific quirks.

Performance Metrics

When predicting physical values using simple linear regression, the latent features consistently outperformed raw data. For instance, using the iCub sensor, the error in predicting pitch was reduced from 4.34 in raw space to 2.92 in latent space. Furthermore, linear classification of curved objects using latent features yielded a nearly perfect confusion matrix.

Control Task Application

To demonstrate practical utility, the latent features were used to stabilize an inverted pendulum. A controller relying solely on these unsupervised features successfully balanced the pole, reaching near-maximum reward within only 30 rollouts.

Conclusion and Future Work

While the VAE effectively separates features, it does not automatically label them; a minimal amount of labeled data is still required if specific physical units (like Newtons) are needed for a task. Future research will focus on more dynamic interactions, such as sliding or vibration, where temporal features play a more critical role. Additionally, determining the optimal minimum dimensionality for the latent space remains an area for further investigation.

Summary updated

Abstract

Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record ground truth reference data. These compact representations can either be used directly in a reinforcement learning based controller or can be used to calibrate the tactile sensor to physical quantities with only a few datapoints. We show the quality of our latent representation by predicting important features and with a simple control task.
BibTeX
@misc{karl2016unsupervised,
  author = {Maximilian Karl and Justin Bayer and Patrick van der Smagt},
  title = {{Unsupervised preprocessing for Tactile Data}},
  year = {2016},
  eprint = {1606.07312},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1606.07312},
}
Figure