Proposal and Evaluation of Visual Haptics for Manipulation of Remote Machine System

Masaki Haruna, Masaki Ogino, Toshiaki Koike-AkinoFrontiers Robotics AI 2020 · DOI · OpenReview

Abstract

Achieving a "sense of oneness" in remote robotic manipulation remains a significant challenge due to the mechanical complexity and cost of physical haptic feedback systems. This paper proposes "Visual Haptics"—the use of simple visual overlays to trigger pseudo-haptic perception—as a means to reduce operator cognitive load. Through a virtual reality study involving a delicate grasping task and EEG monitoring, researchers demonstrated that visual cues at contact points can reduce cognitive burden by 45% and accelerate the learning curve for teleoperation.

Introduction

The demand for remote machine systems is growing across various sectors, from disaster response to operating in hazardous or inaccessible environments. The ultimate goal of these systems is to provide operators with a "sense of oneness," where the robot feels like a natural extension of the body. However, achieving this requires sophisticated sensory feedback to bridge the gap between the operator and the remote environment.

Conceptual framework for remote machine systems and their applications.

The Problem: The Haptic Hardware Hurdle

While visual and auditory feedback have advanced significantly, reproducing the intricate tactile sensations of the human hand is mechanically daunting. Current physical haptic devices, such as force-feedback gloves, are often bulky, expensive, and mechanically complex. This complexity creates a barrier to widespread adoption and can even hinder the operator's natural movement, forcing the brain to work harder to compensate for missing or unnatural sensory data.

A mechanical haptic interface illustrating the complexity the authors aim to simplify.

The Visual Haptics Solution

Instead of physical resistance, the authors propose "Visual Haptics." This technique leverages "pseudo-haptics"—the brain's ability to infer touch from visual stimuli. In their experimental setup, translucent red circles appeared on virtual fingertips the moment they contacted an object. This simple visual "shortcut" aims to replace the need for complex mechanical feedback by providing the brain with the necessary information to process contact intuitively and with less mental effort.

Methods

The study utilized a VR environment where subjects performed a task involving grasping and moving virtual "eggs." Brain activity was monitored using a 32-channel EEG cap. The researchers employed Smooth Coherence Transform (SCoT) to measure "information flow" between brain regions, using it as a proxy for cognitive load. Higher information flow was hypothesized to correlate with the brain struggling to compensate for missing sensory data.

Key Results

  • 45% Reduction in Cognitive Load: The introduction of visual haptics resulted in a nearly halved information flow across alpha, beta, and theta brain waves. This suggests the brain no longer needs to "work overtime" to guess contact timing.
  • Performance and Learning: Success rates in the egg-grasping task improved more rapidly with visual haptics, with a significantly sharper learning curve between the first and third sessions ($p < 0.01$).
  • Objective-Subjective Correlation: A high correlation (0.795) was found between the objective EEG data and the subjects' reported "feeling" of the system, validating EEG as a metric for measuring the sense of oneness.

Conclusion

Visual haptics offer a low-cost, effective alternative to physical force-feedback for remote manipulation. By reducing cognitive load, this approach makes teleoperation more intuitive and easier to learn. Future research will focus on personalizing these visual cues and exploring their effectiveness in more complex, multi-stage robotic tasks to further bridge the gap between human and machine.

Summary updated

Abstract

Remote machine systems have drawn a lot of attention owing to accelerations of virtual reality (VR), augmented reality (AR), and the fifth generation (5G) networks. Despite recent trends of developing autonomous systems, the realization of sophisticated dexterous hand that can fully replace human hands is considered to be decades away. It is also extremely difficult to reproduce the sensilla of complex human hands. On the other hand, it is known that humans can perceive haptic information from visual information even without any physical feedback as cross modal sensation between visual and haptics sensations or pseudo haptics. In this paper, we propose a visual haptic technology, where haptic information is visualized in more perceptual images overlaid at the contact points of a remote machine hand. The usability of the proposed visual haptics was evaluated by subject's brain waves aiming to find out a new approach for quantifying “sense of oneness.” In our proof-of-concept experiments using VR, subjects are asked to operate a virtual arm and hand presented in the VR space, and the performance of the operation with and without visual haptics information as measured with brain wave sensing. Consequently, three results were verified. Firstly, the information flow in the brain were significantly reduced with the proposed visual haptics for the whole α, β, and θ-waves by 45% across nine subjects. This result suggests that superimposing visual effects may be able to reduce the ...
BibTeX
@article{haruna2020proposal,
  author = {Masaki Haruna and Masaki Ogino and Toshiaki Koike-Akino},
  title = {{Proposal and Evaluation of Visual Haptics for Manipulation of Remote Machine System}},
  journal = {Frontiers Robotics AI},
  year = {2020},
  doi = {10.3389/frobt.2020.529040},
}

Loading related content...

Figure