Visuo-Tactile Feedback with Hand Outline Styles for Modulating Affective Roughness Perception

Minju Baeck, Yoonseok Shin, Dooyoung Kim, Hyunjin Lee, Sang Ho Yoon, Woontack Woo

Introduction

In immersive Virtual Reality (VR), tactile feedback is often limited to simple collision-based vibrations that lack emotional depth. This paper introduces a novel visuo-tactile feedback system that alters how users emotionally perceive tactile roughness by changing the visual outline of their virtual hand. By combining fingertip vibrations with dynamic visual cues—such as sharp contours or specific colors—the researchers demonstrate that visual feedback can significantly modulate emotional arousal, valence, and the physical perception of roughness.

The visuo-tactile feedback system combines fingertip vibrations with dynamic hand outline styles.

Problem Statement

While "affective computing" aims to adapt systems to user emotions, designing multimodal feedback that actively shapes these states during active touch remains a challenge. Previous research on crossmodal illusions has primarily focused on changing the appearance of the object being touched. This study explores an underexplored gap: how applying expressive visual cues directly to the user's virtual hand can modulate their emotional response and perceived roughness when paired with fingertip vibrations.

Experimental Design

The researchers developed a VR environment using a Meta Quest 3 headset and a bHaptics TactGlove DK2 to deliver localized fingertip vibrations. Instead of changing the virtual hand's core texture, they rendered a transparent black hand and manipulated its glowing outline during hand-object interactions.

Two main visual parameters were synchronized with haptic feedback:

  1. Line Style: Variations in spatial form (None, Line, Curve, Sharp) and temporal motion (Static, Slow, Fast). For instance, "Curve" used smooth, pulsating noise, while "Sharp" used jagged, sawtooth waveforms.
  2. Color: Red, Green, Blue, and White outlines applied to the static "Line" style.
Experimental conditions for (a) Line Style and (b) Color of the virtual hand outline.

A main study involving 24 participants evaluated how combinations of Haptic × Line Style and Haptic × Color affected self-reported arousal (calm to intense), valence (unpleasant to pleasant), and perceived roughness.

Results and Discussion

Modality Independence

A key finding was that visual and haptic feedback operate independently without statistical interaction effects. This suggests that the brain processes these specific visual and haptic affective cues additively rather than synergistically. For VR designers, this means visual and tactile channels can be tuned separately to target specific emotional outcomes without requiring perfect multimodal congruency.

Visual Compensation

Visual feedback plays a stronger compensatory role when haptic intensity is low. Sharp contours or red colors can stretch the emotional experience into ranges that would normally require much stronger physical vibrations, making this a valuable technique for hardware with limited haptic capabilities.

Shape vs. Color

The shape and motion of the hand outline (Line Style) proved to be a more robust tool than Color. Line style influenced arousal, valence, and the physical perception of roughness. Specifically, "Sharp" outlines significantly increased perceived roughness and arousal while lowering valence compared to smooth "Curve" or static "Line" styles. In contrast, Color only acted as an emotional adjustment layer, affecting valence (with Red eliciting more negative valence) but not arousal or tactile perception.

Conclusion

This research provides a framework for using hand outline styles to modulate affective touch in VR. While the study focused on isolated variables, future work could explore the synergy of combining dynamic contours with color in ecologically valid contexts, such as horror games or therapeutic environments.

Summary updated

BibTeX
@article{baeck2025visuo,
  author = {Minju Baeck and Yoonseok Shin and Dooyoung Kim and Hyunjin Lee and Sang Ho Yoon and Woontack Woo},
  title = {{Visuo-Tactile Feedback with Hand Outline Styles for Modulating Affective Roughness Perception}},
  journal = {IEEE Trans. Vis. Comput. Graph.},
  year = {2025},
  doi = {10.1109/tvcg.2025.3616805},
}
Figure