Abstract
This study investigates how profoundly deaf individuals perceive musical emotions through haptic feedback using a wearable vibrotactile backpack. While haptic technology effectively communicates musical "arousal" (energy and intensity), it faces challenges in conveying "valence" (specific emotional mood). The findings provide critical design guidelines for developing more inclusive audio-haptic technologies for the deaf community.
Introduction
For the profoundly deaf community, music is often physically inaccessible. Traditional assistive technologies like hearing aids and cochlear implants frequently offer poor spectral resolution, making it difficult to discern pitch and timbre—the fundamental building blocks of musical emotion. Recent advances in "tactile-audio" feedback allow users to feel sound waves as vibrations on the skin. However, a significant gap remains in understanding the extent to which profoundly deaf individuals can recognize intended musical emotions through vibrations alone, as previous research has often relied on hearing participants with simulated deafness.
Methodology
The researchers conducted a mixed-methods study involving 16 profoundly deaf participants. The primary hardware used was the SubPac M2x, a commercial vibrotactile backpack that transduces sound frequencies between 1–200Hz into physical vibrations felt on the back.
Participants were presented with 20 piano excerpts, specifically composed by Vieillard et al. (2008) to evoke four distinct emotions: happy, angry, sad, and peaceful. These excerpts control for musical features such as mode, pitch, and tempo. During the quantitative phase, participants rated each excerpt on 9-point Likert scales for arousal (excited vs. bored), emotional valence (positive vs. negative), and confidence. They also completed a multiple-choice emotion identification task.
Key Results and Insights
Arousal vs. Valence
The study found that arousal is significantly easier to perceive through haptics than valence. Participants successfully distinguished "high energy" music (happy and angry) from "low energy" music (sad and peaceful). However, they struggled to differentiate between emotions with similar energy levels, such as sadness and peacefulness.
The "Positive Bias" of Haptics
An interesting finding was the "positive bias" observed in deaf participants. Unlike hearing participants in previous studies who often found haptic vibrations annoying, deaf participants generally perceived the vibrations as positive and engaging. For this community, the sensation of "feeling" the music is inherently rewarding, leading to higher valence ratings across almost all stimuli.
Identification Accuracy
Statistical analysis showed that happy and angry excerpts were correctly identified significantly more often than sad or peaceful ones. Specifically, arousal ratings for Happy (M=6.40) and Angry (M=7.20) were markedly higher than those for Sad (M=3.79) and Peaceful (M=4.09). Qualitative feedback supported these findings, with 11 out of 14 respondents reporting that the backpack created an "engaging and enhanced" experience, even if bass-only vibrations lacked the nuance to distinguish complex musical narratives.
Discussion and Future Work
The research highlights that vibrations serve best as a complement rather than a replacement for auditory input. Participants expressed a strong desire for multimodal experiences—combining haptics with visual cues (like colors or lyrics) and residual auditory input via hearing aids to fill in the "missing pieces."
Future work will aim to address current limitations, including the small sample size and the controlled lab environment. Researchers plan to move into "ecological" settings, such as live concerts, and incorporate physiological metrics like heart rate variability (HRV) to gain more objective measures of emotional response. Additionally, investigating how haptics can convey text-based emotional content from lyrics remains a priority for inclusive design.
Summary updated