EMOTHAW: A novel database for emotional state recognition from handwriting

Laurence Likforman-Sulem, Anna Esposito, Marcos Faúndez-Zanuy, Stéphan Clémençon, Gennaro Cordasco

Abstract

The detection of negative emotional states through daily activities like handwriting offers a promising, non-invasive avenue for mental health screening. This paper introduces EMOTHAW (EMOTion recognition for HAndWriting), the first publicly available database designed to correlate handwriting dynamics with emotional states. Utilizing a dataset of 129 participants whose states of anxiety, depression, and stress were assessed via the Depression Anxiety Stress Scales (DASS), we analyze handwriting features captured through digitizing tablets. By employing Random Forest classifiers, we demonstrate that these emotional states can be identified with accuracies ranging from 60% to 71%, highlighting the significance of both on-paper and in-air pen movements.

Introduction

Proactive mental healthcare relies on the early detection of negative emotional states. However, traditional clinical interviews can be resource-intensive and invasive. While handwriting analysis has been successfully applied to the screening of neurodegenerative diseases like Parkinson’s and Alzheimer’s, its application in identifying "softer" psychological states like stress or depression is a relatively new frontier. The primary obstacle has been a lack of standardized, labeled data; most existing databases focus on biometrics or text recognition. EMOTHAW fills this gap by providing a dataset that links digitized handwriting samples with validated psychological scores.

The EMOTHAW Database

The database consists of samples from 129 university students (71 female, 58 male) aged 21–32. To establish ground truth, participants completed the DASS-42, a validated 42-item self-report questionnaire measuring depression, anxiety, and stress.

Participants performed seven varied tasks using a Wacom digitizing tablet and an "Inkpen," which allows for writing on physical paper while simultaneously recording digital data. The tasks included:

  • Drawing pentagons, a house, a clock, and circles.
  • Copying words in handprint.
  • Copying a phonetically complete sentence in cursive writing.

Records captured seven primary parameters: X/Y coordinates, timestamps, pen status (on-paper vs. in-air), azimuth, altitude, and pressure.

Methodology

The research team implemented a machine learning pipeline focused on feature extraction and classification:

  1. Feature Extraction: The study focused on 20 specific features across five tasks, categorized into timing-based features (e.g., time spent in-air vs. on-paper) and ductus-based features (e.g., the number of strokes and the specific way a stroke is drawn).
  2. Classification: The researchers utilized Random Forests, an ensemble learning method. This approach was selected for its predictive power and its ability to rank features. By measuring the increase in "Out-of-Bag" (OOB) error when a feature is permuted, the team identified which handwriting characteristics were most indicative of each emotional state.

Key Insights

The "In-Air" Phenomenon

A significant finding of the study is that in-air movements—the moments when the pen moves between strokes without touching the paper—are as informative as the writing itself. For example, participants exhibiting depression showed longer in-air pauses, a behavior that mirrors the "speech pauses" often observed in the vocal analysis of depressed individuals.

Task Specificity

The expression of emotional states varies by task. Depression was most clearly identified through drawing tasks (clock, house, pentagons). Conversely, anxiety and stress were better revealed through a combination of both drawing and writing tasks (cursive and handprint).

Ductus as a Behavioral Mirror

The "ductus"—the order, speed, and direction of strokes—serves as a behavioral reflection of the writer's state. For stressed and depressed participants, in-air movements were often more erratic or significantly slower than those in the "normal" control group.

Results

The Random Forest models were evaluated using leave-one-out cross-validation, yielding the following accuracies:

  • Depression: 71.2%
  • Stress: 60.2%
  • Anxiety: 60.0%

While these results represent a baseline for a first-of-its-kind dataset, they are significantly above chance and suggest that handwriting dynamics are a viable marker for psychological health.

Limitations and Future Work

The current study is limited by the homogeneity of its cohort (university students aged 21–32). Because handwriting and emotional expression evolve with age, future research must test these models on children and the elderly.

Proposed future directions include:

  • Advanced Kinematics: Incorporating fluidity and velocity changes to extract more complex features.
  • Pressure Analysis: Deeper investigation into pen pressure, which is expected to significantly improve stress detection.
  • Broader Demographics: Expanding the database to include a more diverse range of ages and backgrounds to create a universal screening tool.

Summary updated

Abstract

The detection of negative emotions through daily activities such as handwriting is useful for promoting well-being. The spread of human-machine interfaces such as tablets makes the collection of handwriting samples easier. In this context, we present a first publicly available handwriting database which relates emotional states to handwriting, that we call EMOTHAW. This database includes samples of 129 participants whose emotional states, namely anxiety, depression and stress, are assessed by the Depression Anxiety Stress Scales (DASS) questionnaire. Seven tasks are recorded through a digitizing tablet: pentagons and house drawing, words copied in handprint, circles and clock drawing, and one sentence copied in cursive writing. Records consist in pen positions, on-paper and in-air, time stamp, pressure, pen azimuth and altitude. We report our analysis on this database. From collected data, we first compute measurements related to timing and ductus. We compute separate measurements according to the position of the writing device: on paper or in-air. We analyse and classify this set of measurements (referred to as features) using a random forest approach. This latter is a machine learning method [2], based on an ensemble of decision trees, which includes a feature ranking process. We use this ranking process to identify the features which best reveal a targeted emotional state. We then build random forest classifiers associated to each emotional state. Our results, obtained from cross-validation experiments, show that the targeted emotional states can be identified with accuracies ranging from 60% to 71%.
BibTeX
@article{likformansulem2017emothaw,
  author = {Laurence Likforman-Sulem and Anna Esposito and Marcos Faúndez-Zanuy and Stéphan Clémençon and Gennaro Cordasco},
  title = {{EMOTHAW: A novel database for emotional state recognition from handwriting}},
  journal = {IEEE Trans. Hum. Mach. Syst.},
  year = {2017},
  url = {https://openreview.net/forum?id=vzz1PUR3Gm},
}
Figure