Handwriting Analysis on the Diaries of Rosamond Jacob

Sharmistha S. Sawant, Saloni D. Thakare, Derek Greene, Gerardine Meaney, Alan F. Smeaton

TL;DR

This paper introduces a novel methodology for analyzing historical handwritten diaries by combining handwriting analysis with sentiment analysis. By developing canonical forms of common words and correlating handwriting patterns with sentiment scores, the authors aim to uncover insights into the writer's emotional state and the impact of historical events.

What Problem Does This Paper Address?

Historians and social scientists often rely on personal documents like diaries to understand past events and individual experiences. However, manually sifting through vast collections of handwritten material to identify patterns, track sentiment, or correlate them with specific events is incredibly time-consuming and prone to human bias. This paper tackles the challenge of automating this process by developing computational methods to analyze both the physical characteristics of handwriting and the emotional tone of the text within historical diaries, specifically focusing on the diaries of Rosamond Jacob, an Irish activist. The goal is to unlock deeper, data-driven insights into historical narratives and the personal impact of significant events.

Methods

The authors propose a multi-stage approach:

  • Image Processing and Word Segmentation: They begin by cleaning and preparing scanned diary images. This involves cropping borders, grayscaling, thresholding, and dilation to enhance text clarity. Crucially, they employ contour detection to segment individual words and create bounding boxes for each.
  • Canonical Form Generation: To compare handwriting over time, they identify the 5 most frequently occurring words (excluding "and" due to its frequent substitution with "&"). For each of these words, they generate a "canonical form" by clustering multiple occurrences of the word within a diary entry and taking the average of the largest cluster. This represents a standardized version of the word's appearance for that specific entry.
  • Handwriting Similarity Measurement: They evaluate the similarity between these canonical forms using three metrics:
  • Mean Squared Error (MSE): A basic pixel-level comparison.
  • Structural Similarity Index Measure (SSIM): A more perceptually-based metric that considers luminance, contrast, and structure.
  • Dynamic Time Warping (DTW): An algorithm well-suited for comparing time series or sequences of varying lengths, applied here to normalized image data. DTW proved to be the most effective for capturing similarity among handwritten words.
  • Sentiment Analysis: They utilize the Hedonometer, a dictionary-based tool that assigns a "happiness score" to text. To account for the historical nature of the diaries, they use pre-existing Hedonometer scores rather than relying on contemporary sentiment analysis tools that might struggle with older vocabulary.
  • Correlation Analysis: Finally, they correlate the generated canonical forms of handwriting with the sentiment scores of the corresponding diary entries, and also analyze sentiment around specific named entities.

Datasets

The primary dataset used is a collection of digitized diary entries from Rosamond Jacob (1888-1960), an Irish writer and political activist. The study focuses on a subset of these diaries from the years 1917-1921, with specific analysis performed on entries mentioning the named entity "Dorothy" within the period 1917-1922. No new benchmarks were introduced; rather, existing techniques were applied to this unique historical dataset.

Key Insights and Ideas

  • DTW as a Superior Handwriting Similarity Metric: The research strongly suggests that Dynamic Time Warping (DTW) is more effective than MSE or SSIM for capturing nuanced similarities in handwritten words, especially when dealing with variations in size and stroke.
  • Handwriting as a Window into Emotion: The study explores the hypothesis that handwriting style can correlate with the writer's emotional state and the topics they are discussing, offering a potential avenue for inferring mood from physical writing.
  • Sentiment Analysis for Historical Context: Applying sentiment analysis to historical texts, even with potential vocabulary mismatches, can provide valuable insights into the emotional landscape of the time and the writer's personal reactions to events.
  • Named Entities as Focal Points for Analysis: Focusing on diary entries that mention specific individuals (like "Dorothy") allows for a more targeted investigation into how personal relationships and interactions might influence both handwriting and expressed sentiment.

Key Results

  • DTW's Performance: DTW consistently captured more similarity among instances of the same handwritten words compared to MSE and SSIM.
  • Sentiment Volatility: The analysis of Rosamond Jacob's diaries revealed significant volatility in sentiment scores, with notable dips corresponding to periods of intense political events, such as the lead-up to the Irish Civil War and reports of violence against women.
  • Entity-Specific Sentiment: Diary entries mentioning the named entity "Dorothy" showed sentiment scores that were generally higher than the average, with some standout entries in 1921 exhibiting particularly high scores.
  • No Strong Handwriting-Sentiment Correlation Found (Yet): While the methodology was established, the paper notes that a strong, direct correlation between handwriting characteristics and sentiment was not definitively established within the analyzed segment. This is attributed to factors like image quality and the complexity of human emotion.

Limitations and Future Work

  • Limited Data Segment: The current analysis is based on a relatively small segment (5 years) of Rosamond Jacob's diaries.
  • Image Quality: Some diary entries suffered from faint ink and low brightness, which affected the quality of image segmentation and potentially the accuracy of handwriting analysis.
  • Vocabulary Mismatch for Sentiment: While Hedonometer was used, the paper acknowledges the potential for vocabulary mismatches with texts over a century old.
  • Future Work: The authors propose extending the analysis to a larger time range of diary entries, focusing on handwriting changes around specific topics or named entities, and further investigating the applicability of the Hedonometer vocabulary to older texts. They also aim to establish stronger correlations between handwriting and sentiment.

Summary updated

Abstract

Handwriting is an art form that most people learn at an early age. Each person's writing style is unique with small changes as we grow older and as our mood changes. Here we analyse handwritten text in a culturally significant personal diary. We compare changes in handwriting and relate this to the sentiment of the written material and to the topic of diary entries. We identify handwritten text from digitised images and generate a canonical form for words using shape matching to compare how the same handwritten word appears over a period of time. For determining the sentiment of diary entries, we use the Hedonometer, a dictionary-based approach to scoring sentiment. We apply these techniques to the historical diary entries of Rosamond Jacob (1888-1960), an Irish writer and political activist whose daily diary entries report on the major events in Ireland during the first half of the last century.
BibTeX
@inproceedings{sawant2023handwriting,
  author = {Sharmistha S. Sawant and Saloni D. Thakare and Derek Greene and Gerardine Meaney and Alan F. Smeaton},
  title = {{Handwriting Analysis on the Diaries of Rosamond Jacob}},
  booktitle = {Cbmi},
  year = {2023},
}
Figure