PsyMo: A Dataset for Estimating Self-Reported Psychological Traits From Gait

Adrian Cosma, Emilian RadoiWACV 2024

TL;DR

This paper introduces PsyMo, a novel, large-scale dataset for estimating psychological traits from human gait. By capturing walking sequences of 312 individuals alongside their self-reported personality, fatigue, and mental health data, PsyMo enables interdisciplinary research into the embodied manifestation of psychological states in movement. The dataset also serves as a valuable benchmark for traditional gait recognition tasks.

What Problem Does This Paper Address?

While gait analysis has seen significant advancements in identity recognition, its potential for inferring psychological states remains largely unexplored. Existing research often relies on small, private datasets or intrusive body-attached sensors. This gap limits our ability to understand how internal psychological states, such as personality, stress, or fatigue, manifest in observable human behavior like walking. Such an understanding could have applications in areas like mental health monitoring, occupational safety, and personalized marketing. The authors aim to bridge this gap by creating a comprehensive, publicly available dataset that facilitates research into the connection between gait and psychological traits.

Methods

The core contribution of this paper is the creation and release of the PsyMo dataset. The dataset was collected under controlled laboratory conditions designed to mimic real-world surveillance scenarios, using three synchronized consumer-grade cameras. Participants walked in seven variations (normal, changing clothes, slow, fast, with a bag, walking while texting, and walking while talking on the phone) from six viewpoints.

Crucially, the dataset is enriched with psychometric data. Participants completed six validated questionnaires, yielding 17 psychological traits related to personality (Big Five Inventory), self-esteem (Rosenberg Self-Esteem), aggression (Buss-Perry Aggression Questionnaire), fatigue (Occupational Fatigue Exhaustion/Recovery Scale), and mental health (Depression Anxiety Stress Scale, General Health Questionnaire).

To ensure privacy, raw RGB videos are not released. Instead, the authors provide processed data including:

  • Silhouettes: Extracted using a pretrained Hybrid Task Cascade (HTC) model.
  • 2D/3D Skeletons: Obtained using AlphaPose for 2D and CLIFF for 3D estimation.
  • 3D SMPL Human Meshes: Predicted using CLIFF.

The paper also proposes two evaluation protocols for psychological trait estimation: run-level (evaluating each sequence independently) and subject-level (evaluating the ability to identify traits for an entire subject across multiple sequences).

Datasets

Newly Introduced Benchmark: * PsyMo: A dataset containing 14,976 walking sequences from 312 subjects, annotated with 17 psychological traits, gender, age, height, and weight. It features 7 walking variations and 6 viewpoints, captured using consumer-grade surveillance cameras. The dataset is anonymized and released with processed silhouettes, 2D/3D skeletons, and 3D SMPL human meshes.

Existing Datasets for Comparison/Context: * CASIA-B: A controlled gait dataset used as a benchmark for gait recognition, with variations in walking and viewpoints. * FVG: Another controlled gait dataset. * GREW, Gait3D, DenseGait: Datasets collected "in the wild" with less controlled variations.

Key Insights and Ideas

  • Embodiment Hypothesis: The research is grounded in the theory of embodiment, which posits a biofeedback loop between the psyche and physical actions. This suggests that psychological states can be reflected in how we move.
  • Gait as a Window to the Psyche: The paper strongly advocates for gait as a rich, yet underutilized, modality for inferring psychological traits, offering a non-intrusive alternative to traditional methods.
  • Real-World Simulation: The use of consumer-grade cameras and specific capture settings aims to bridge the gap between controlled laboratory studies and the complexities of real-world surveillance, making the dataset more relevant for practical applications.
  • Dual-Task Walking: The inclusion of "walking while texting" and "walking while talking on the phone" as walking variations is a novel contribution, reflecting common real-world scenarios that can significantly alter gait.
  • Multi-Modal Data Fusion Potential: The dataset provides multiple representations of gait (silhouettes, skeletons, meshes), opening avenues for research into multi-modal fusion for improved trait estimation.

Key Results

The paper presents baseline results for psychological trait estimation using various modalities and models.

  • Silhouettes vs. Skeletons: Preliminary results suggest that mental distress and fatigue are more easily discernible from silhouettes than from 2D skeletons.
  • SMPL-based Models: SMPLGait, which combines SMPL parameters and silhouettes, performed reasonably well in run-level evaluation but showed less consistency in subject-level prediction.
  • GaitFormer: This transformer-based architecture, which processes whole skeletons, demonstrated significant improvements in subject-level evaluation, outperforming other models across modalities.
  • Subject-Level Performance: Subject-level evaluation generally yielded higher results, indicating that models capable of processing sets of runs are crucial for robust trait estimation.

Limitations and Future Work

Limitations: * Researcher Presence: The authors acknowledge that the presence of a researcher in the lab might subtly influence participants' walking patterns. * Dataset Bias: The dataset primarily consists of university students from Romania, which may limit its generalizability to other age groups, cultures, and races.

Future Work: * Fine-tuning Pretrained Models: The recommended approach for using PsyMo is to fine-tune general-purpose models pretrained on larger, more diverse datasets. * Exploiting Multi-Modal Data: Future research can leverage the multiple gait representations (silhouettes, skeletons, meshes) and the natural correlations between psychological subscales for multi-task learning. * Developing Novel Architectures: The dataset's complexity encourages the development of new, higher-level architectures capable of processing multiple walking sequences for more accurate trait estimation. * Dataset Expansion: The authors plan to add more data modalities and update semantic features with improved models over time.

Summary updated

BibTeX
@inproceedings{cosma2024psymo,
  author = {Adrian Cosma and Emilian Radoi},
  title = {{PsyMo: A Dataset for Estimating Self-Reported Psychological Traits From Gait}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure