Flexible Piezoelectric Tactile Sensing System for Intelligent Object Recognition

Razan Khalifeh, Mohamad Yaacoub, Christian Gianoglio, Moustafa Saleh, Maurizio Valle, Yahya Abbass

This study presents a flexible, distributed piezoelectric tactile sensing system—essentially an "electronic skin"—designed to give robots the ability to identify objects the moment they touch them. By combining screen-printed sensor arrays with a novel spatio-temporal heatmap representation and deep learning, the researchers achieved near-perfect recognition accuracy on an embedded edge platform with a total latency of just 101 milliseconds.

TL;DR

Researchers developed a flexible piezoelectric "e-skin" that allows robots to identify objects within 101ms of contact with over 99% accuracy. By converting tactile signals into spatio-temporal heatmaps and processing them with a 2D-CNN on a low-power microcontroller, the system enables fast, energy-efficient, and reactive robotic grasping.

What Problem Does This Paper Address?

While computer vision has revolutionized robotics, it struggles with transparent objects, occlusions during grasping, and perceiving intrinsic material properties like stiffness or texture. Tactile sensing fills this gap, but existing solutions often face a trade-off: they are either too rigid to fit on soft grippers, too computationally heavy for real-time use, or fail to provide reliable recognition during the critical "early stage" of a grasp.

The authors tackle the challenge of creating a sensing system that is flexible, high-resolution, and capable of performing complex object recognition directly on "edge" hardware (like a robot's onboard microcontroller) without sacrificing speed or accuracy.

Methods

The team developed a fully screen-printed flexible sensor array using P(VDF-TrFE), a piezoelectric polymer. These arrays were integrated into custom-molded silicone "sensing caps" (using "Dragon Skin" material) and mounted onto the grippers of a TIAGo service robot.

The sensing system integration: from single sensor structure to mounting on the TIAGo robot.

The core of their software contribution lies in comparing three distinct machine learning pipelines:

  1. Feature-Based Approach (FBA): Manual extraction of statistical features (like energy and peak-to-peak values) followed by a Support Vector Machine (SVM).
  2. End-to-End Approach (EEA): Feeding raw tactile signals directly into 1D Convolutional Neural Networks (1D-CNN) or Convolutional Recurrent Neural Networks (C-RNN).
  3. Heatmap Approach (HMA): The most novel method, which transforms the temporal window of tactile data into a 2D spatial matrix (a "tactile image") based on the physical layout of the sensors. This is then processed by a 2D-CNN.

Datasets

The researchers collected a representative dataset by having the TIAGo robot grasp six daily-life objects:

  • A glass bottle
  • A TV remote controller
  • A half-filled plastic water bottle
  • Three 3D-printed fruits (Strawberry, Rigid Apple, and Semi-Soft Apple) from the YCB Object Set.

They recorded 300 trials (50 per object). To ensure the model learned robust features rather than just memorizing specific sequences, they partitioned the data by entire trials rather than individual samples, preventing "temporal leakage."

Key Insights and Ideas

  • Heatmaps Outperform Raw Signals: Treating tactile data as a spatio-temporal image (HMA) proved superior to sequence-based models. This suggests that the spatial relationship between sensors is just as critical as the timing of the signal for identifying shapes and textures.
  • The "Early Touch" Advantage: Piezoelectric sensors are inherently dynamic—they respond to changes in force. This makes them exceptionally well-suited for "early-stage" recognition (the first 100ms of contact) where the signal is most informative.
  • Sensor Density Matters: The study systematically evaluated the impact of sensor count. While 16 sensors provided strong results, increasing to 32 sensors pushed the F1-score to its peak, demonstrating the value of high-density "e-skin."
  • Edge-Ready Efficiency: The best-performing 2D-CNN model is remarkably lightweight, requiring only 7.65 kB of flash memory. This allows it to run on a standard ARM Cortex-M4F microcontroller with negligible energy consumption (63.8 µJ per inference).

Key Results

The Heatmap Approach (HMA) with a 2D-CNN was the clear winner:

  • Accuracy: Achieved an F1-score of 0.9903 using all 32 sensors and a 100ms temporal window.
  • Latency: The end-to-end system latency was measured at ~101 ms. Crucially, 100ms of this is simply the time required to collect the data window; the actual inference takes less than 1ms.
  • Power: The model consumes only 72.3 mW during operation, making it ideal for battery-powered mobile robots.
Performance comparison of different models and sensor counts. The HMA (2D-CNN) consistently leads.

Limitations and Future Work

While the results are impressive, the authors note a few constraints:

  • Object Diversity: The system was tested on a relatively small set of six objects. Future work will need to scale this to hundreds of objects with more varied geometries.
  • Grasp Strategies: The current study used a predefined grasp-hold-release sequence. Real-world scenarios involve dynamic manipulation, slippage, and varied approach angles.
  • Generalization: The system was validated on a single robotic platform. Testing on different types of grippers (e.g., multi-fingered anthropomorphic hands) is a logical next step.

By proving that high-accuracy tactile recognition can happen almost instantaneously on low-power hardware, this research moves us one step closer to robots that can "feel" and react to their environment as naturally as humans do.

Summary updated

Abstract

Tactile sensing provides information about object properties that can improve robotic grasping and manipulation. However, accurate object recognition during the early stages of grasping remains challenging, particularly under strict real-time constraints. This study presents a flexible, distributed piezoelectric tactile sensing system for early-stage object recognition. The system was experimentally evaluated using six everyday objects and three recognition approaches: feature-based classification using support vector machines (SVMs), raw-signal classification using 1D convolutional neural networks (1D-CNNs) and convolutional recurrent neural networks (C-RNNs), and spatiotemporal heatmap classification using a 2D-CNN. The effects of sensor count and temporal window size on recognition accuracy and real-time feasibility were systematically investigated. Increasing the number of sensing elements improved recognition performance across all approaches. The best-performing method, based on spatiotemporal heatmaps and a 2D-CNN, achieved F1-scores of 0.9796, 0.9846, and 0.9903 using 16, 24, and 32 sensors, respectively. The optimal configuration, comprising 32 sensors and a 100-ms temporal window (50 samples), was deployed on an embedded edge platform, achieving an end-to-end recognition latency of approximately 101 ms from contact onset and an energy consumption of 63.8 microjoules per inference. These results demonstrate the potential of distributed piezoelectric tactile sensing for accurate, energy-efficient, and real-time object recognition during robotic grasping.
BibTeX
@misc{khalifeh2026flexible,
  author = {Razan Khalifeh and Mohamad Yaacoub and Christian Gianoglio and Moustafa Saleh and Maurizio Valle and Yahya Abbass},
  title = {{Flexible Piezoelectric Tactile Sensing System for Intelligent Object Recognition}},
  year = {2026},
  eprint = {2611.07779},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.07779},
}
Figure