VET: A Visual-Electronic Tactile System for Immersive Human-Machine Interaction

Wenbo Ding (丁文伯), Cong Zhang, Yisheng Yang, Shilong Mu, Chuqiao Lyu, Shoujie Li, Xinyue Chai, Wenbo Ding

TL;DR

This paper introduces the Visual-Electronic Tactile (VET) system, a novel human-machine interface that seamlessly integrates vision-based tactile sensing with electrical stimulation for haptic feedback. By overcoming interference issues common in traditional haptic systems, VET enables more immersive and precise bidirectional tactile interactions for applications like virtual reality and robotic teleoperation.

What Problem Does This Paper Address?

Achieving truly immersive human-machine interaction (HMI) requires sophisticated tactile input and output capabilities. While vision-based tactile sensors (VBTS) offer rich information acquisition for tasks like robotic manipulation, integrating traditional haptic feedback mechanisms (like force or vibration) with VBTS presents a significant challenge. These traditional methods can interfere with the delicate sensing mechanisms of VBTS, reducing data accuracy and hindering the development of effective closed-loop control systems. This interference is particularly problematic in high-precision applications. The paper addresses this gap by proposing a system that uses electrical stimulation for haptic feedback, which can be integrated onto the VBTS surface without compromising its sensing performance.

Methods

The core innovation of the VET system lies in its integrated design and fabrication process.

  • System Architecture: VET combines a vision-based tactile sensor (VBTS) with an electrical stimulation film. The VBTS captures multi-dimensional tactile data (force, contour, motion) through optical means. The electrical stimulation film, integrated directly onto the VBTS surface, provides haptic feedback by stimulating neural pathways in the user's skin.
  • Fabrication Process: A key contribution is the screen-printing preparation process used to integrate the electrical stimulation film onto the VBTS. This method allows for a seamless union, eliminating the interference issues associated with separate sensor and actuator components. The fabrication involves:
  • VBTS Fabrication: Standard processes involving substrate preparation, thin film deposition, patterning, and protective layers.
  • Electrode Film Fabrication: Screen printing of conductive ink onto a stretchable substrate (TPU), followed by mechanical stretching to activate conductivity.
  • Laser Perforation & Bonding: Laser cutting to create electrode areas and then bonding the prepared film to the VBTS using silicone adhesive.
  • Electrical Stimulation: The system employs a programmable dual-channel stimulator. It allows for precise control over stimulation parameters like frequency and intensity. The paper notes that frequencies between 0.5 Hz and 100 Hz are relevant for stimulating Meissner corpuscles and Merkel cells, with 50 Hz being used in experiments. The back grounding electrode configuration is chosen for its practicality in wearable devices.
  • Spatial Control: The polarity of electrical pulses is used to control the perceived location of stimulation on the fingertip, allowing for targeted feedback. Alternating positive and negative pulses concentrate the sensation at the contact point.

Datasets

Not discussed in the paper. The experiments described appear to be conducted using the developed VET system itself, with human participants providing subjective feedback and objective measurements.

Key Insights and Ideas

  • Seamless Integration: The screen-printing fabrication method is a critical enabler, allowing for the co-location of sensing and stimulation without mutual interference. This is a significant step beyond systems that require separate, potentially disruptive, components.
  • Targeted Haptic Feedback: Electrical stimulation offers a precise and diverse feedback modality, capable of conveying information prompts and intensity alerts without physically disturbing the tactile sensor.
  • Spatial Sensitivity of Skin: The experiments reveal distinct spatial sensitivity variations across different finger regions, with the fingertip and central zones being most sensitive. This understanding is crucial for designing effective and nuanced electrotactile feedback.
  • Enhanced Immersion: The VET system demonstrably improves immersion in applications like VR flight simulations and robotic teleoperation by providing context-aware, multi-modal tactile feedback.

Key Results

  • No Interference: The study shows that the integrated electrical stimulation film does not significantly impact the tactile sensing performance of the VBTS, with tactile images remaining largely consistent.
  • Spatial Sensitivity: Fingertip and central finger regions exhibit the highest sensitivity to electrical stimulation, while left and right zones show lower sensitivity.
  • Application Performance:
    • In a VR flight simulation, users reported significantly enhanced immersion and gameplay enjoyment when experiencing electrotactile feedback for environmental hazards (e.g., ice, fire).
  • In robotic arm teleoperation, the VET system allows for more precise control over grasping force when interacting with objects of varying softness and hardness, enabling operators to fine-tune grip strength in real-time.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that electrode durability and user-specific sensitivity are ongoing challenges in electrotactile technology.
  • While the system is compact, further miniaturization and wireless connectivity could enhance portability for real-world deployment.
  • Future Work:
    • Improving system adaptability and accessibility.
  • Incorporating multidimensional tactile sensing to expand user input capabilities.
  • Developing adaptive algorithms to personalize electrical stimulation for diverse user needs.
  • Exploring broader applications in human-machine collaboration and robotics.
  • Advancements in hardware miniaturization and wireless connectivity.

Summary updated

Abstract

In the pursuit of deeper immersion in human-machine interaction, achieving higher-dimensional tactile input and output on a single interface has become a key research focus. This study introduces the Visual-Electronic Tactile (VET) System, which builds upon vision-based tactile sensors (VBTS) and integrates electrical stimulation feedback to enable bidirectional tactile communication. We propose and implement a system framework that seamlessly integrates an electrical stimulation film with VBTS using a screen-printing preparation process, eliminating interference from traditional methods. While VBTS captures multi-dimensional input through visuotactile signals, electrical stimulation feedback directly stimulates neural pathways, preventing interference with visuotactile information. The potential of the VET system is demonstrated through experiments on finger electrical stimulation sensitivity zones, as well as applications in interactive gaming and robotic arm teleoperation. This system paves the way for new advancements in bidirectional tactile interaction and its broader applications.
BibTeX
@inproceedings{2025vet,
  author = {Wenbo Ding (丁文伯) and Cong Zhang and Yisheng Yang and Shilong Mu and Chuqiao Lyu and Shoujie Li and Xinyue Chai and Wenbo Ding},
  title = {{VET: A Visual-Electronic Tactile System for Immersive Human-Machine Interaction}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2025},
  doi = {10.1109/iros60139.2025.11247401},
}
Figure