This paper introduces BODYinTRANSIT-Lab, a comprehensive multimodal dataset designed to study how our perception of our own bodies—and the emotions tied to that perception—can be reshaped through technology. By manipulating the sounds of a person's own footsteps in real-time, researchers induced "Body Transformation Experiences" (BTEs) and captured the resulting changes in movement, physiology, and subjective feeling.
TL;DR
The authors present BODYinTRANSIT-Lab, a first-of-its-kind multimodal dataset linking controlled sensory manipulation (the "Footsteps Illusion") with full-body kinematics, physiology, and rich subjective annotations from 101 participants. It provides a benchmark for using deep learning to infer a person's perceived body weight and affective state directly from their walking patterns.
What Problem Does This Paper Address?
Our mental representation of our bodies is not fixed; it is a dynamic construct continuously updated by sensory input. Interactive technologies can deliberately manipulate this input to induce BTEs—temporary changes in how we perceive our body's size, weight, or capabilities.
While BTEs have profound implications for health (e.g., treating eating disorders or chronic pain) and well-being, we lack the computational models to understand them fully. Most existing affective datasets focus on "acted" emotions or responses to external stimuli (like videos). There is a significant gap in data that connects closed-loop sensory manipulation (where your own actions generate the feedback) with both objective bodily signals and subjective experience.
Methods
The researchers employed the Footsteps Illusion using a custom system called SoniWeight Shoes. As participants walked, the system captured their footstep sounds, manipulated their frequency spectrum in real-time, and played them back through headphones.
- High-frequency (HF) boost: Intended to make participants feel lighter/slimmer.
- Low-frequency (LF) boost: Intended to make participants feel heavier/larger.
To capture the response, the team used a sophisticated sensor montage:
- Kinematics: A Rokoko Smartsuit Pro II recorded 306 channels of full-body motion data at 100Hz.
- Physiology: BITalino sensors captured EMG (muscle activity), ECG (heart rate), EDA (skin conductance), and respiration at 1000Hz.
- Subjective Reporting: This is a standout feature of the dataset. It includes within-trial verbal ratings, post-trial affective scales (valence, arousal, dominance), 3D avatar adjustments to visualize perceived size, and digital "body maps" where participants marked where they felt changes.
Datasets
The BODYinTRANSIT-Lab dataset includes 101 adults (~16 hours of data). Crucially, the recruitment was stratified by eating-disorder symptomatology and physical activity levels, as these traits are known to influence how individuals respond to body illusions.
The authors compare their work to existing benchmarks like Emilya (acted emotions) and EmoPain (pain-related movement), noting that BODYinTRANSIT-Lab is unique in its combination of controlled action-dependent feedback and high-density spatial/perceptual annotations.
Key Insights and Ideas
- Experience vs. Stimulus: A key "aha moment" was that the deep learning models were better at predicting a participant's reported experience (how heavy they felt) than the actual auditory condition they were assigned. This suggests that movement patterns are a more direct reflection of internal subjective states than the external manipulation itself.
- The Importance of Individual Context: By stratifying the participants, the authors acknowledge that a "one-size-fits-all" model of body perception is insufficient. Factors like body-related attitudes and fitness levels significantly modulate the effect of the illusion.
- Spatiality of Feeling: The use of body maps reveals that BTEs aren't just "global" feelings; people experience changes in specific regions. This spatial data offers a new dimension for affective computing beyond simple scalar ratings.
Key Results
The authors established a computational benchmark using a deep learning architecture with attention pooling to fuse data from 17 different body locations.
| Task | Balanced Accuracy (BA) |
|---|---|
| Auditory Condition Classification | 36.6% |
| Relative Perceived Weight | 44.5% |
| Relative Affective Valence | 45.9% |
| (Note: Chance level for these 3-class tasks is 33.3%) |
The attention profiles (shown in Figure 6) indicated that the model relied most heavily on sensors located on the feet and hands to make its predictions, which aligns with the nature of a walking-based illusion.
Limitations and Future Work
- Generalization: The current benchmark uses within-session splits. Future work needs to test if these models can generalize to entirely new participants or different sessions.
- Demographics: The sample was primarily young adults in Spain; broader cultural and age-related studies are needed.
- Ethics: The authors explicitly discuss the ethical risks of "body-perception technologies." They warn against using these models to push users toward "normative" or "ideal" body types, advocating instead for systems that support user autonomy and diverse models of embodiment.
- Multimodality: The initial benchmark only used kinematics. Integrating the physiological data (ECG, EDA, etc.) is a promising next step to improve prediction accuracy.
Summary updated