Estimation of Happiness Changes through Longitudinal Analysis of Employees’ Texts

Junko Hayashi, Kazuhiro Ito, Masae Manabe, Yasushi Watanabe, Masataka Nakayama, Yukiko Uchida, Shoko Wakamiya, Eiji AramakiACL 2024

TL;DR

This paper introduces a novel approach to track individual happiness fluctuations in the workplace using longitudinal free-text diary entries. By leveraging a BERT-based model, they demonstrate the feasibility of estimating daily happiness levels, offering a more granular alternative to traditional surveys and paving the way for timely workplace interventions.

What Problem Does This Paper Address?

Measuring and understanding employee well-being is crucial for workplace performance, productivity, and retention. While existing methods like questionnaires can assess general happiness, they often fail to capture short-term, daily changes in an individual's emotional state. This limitation hinders the ability to identify immediate issues and implement timely interventions. The challenge lies in collecting and analyzing granular, longitudinal data that reflects these daily fluctuations. Previous text-based happiness estimation studies have largely focused on aggregate-level analysis rather than individual trajectories, leaving a gap in understanding personal happiness dynamics over time.

Methods

The core of this study is a BERT-based estimation model designed to predict happiness scores from free-text diary entries. The authors adapted a pre-trained BERT model (specifically, the Tohoku University pre-trained BERT) for sequence classification and then fine-tuned it as a regression model. A key aspect of their approach is the simplification of the model to estimate happiness using only one diary entry per day, making it more practical for longitudinal tracking. This contrasts with more complex models that might require extensive context. For evaluation, they employed the Pearson correlation coefficient to measure the agreement between the model's estimated happiness scores and the users' self-reported happiness scores.

Datasets

The study utilized two datasets:

  • Company A Data: This dataset, previously used in another study, served as the training data for the happiness estimation model. It comprised daily reports from 94 employees over two months (September 1 to October 31, 2022), yielding 1,728 data points.
  • Company B Data: This dataset was used for evaluation and testing of the trained model. It consisted of daily reports from 11 employees over six months (December 19, 2022, to May 18, 2023), resulting in 652 data points. This approach of testing on a completely different population under strict conditions aims to verify the model's generality.

Additionally, personality traits (cooperativeness and autonomy) were collected from participants in Company B to investigate their influence on model performance.

Key Insights and Ideas

  • Feasibility of Daily Happiness Tracking: The study successfully demonstrates that free-text diary entries can be used to estimate daily happiness levels, providing a more dynamic view than traditional surveys.
  • Individual Variability is Key: The model's performance varies significantly across individuals, highlighting that personal writing styles and underlying factors influencing happiness are crucial for accurate estimation.
  • Textual Features Matter: The analysis suggests that longer sentences and greater lexical diversity in diaries are associated with better happiness estimation performance, potentially because they offer richer information.
  • Personality Traits Influence Estimation: While not always statistically significant due to small sample sizes, there's a trend indicating that personality traits like cooperativeness might play a role in how well the model can estimate happiness from an individual's text.

Key Results

  • The BERT-based model achieved a significant correlation between estimated and self-reported happiness scores for 9 out of 11 participants in Company B, with correlation coefficients of 0.4 or higher.
  • For one user (U1), the self-reported happiness score remained constant, making correlation calculation impossible. Another user (U3) showed a negative correlation, which the authors attribute to their writing style.
  • The study found a positive trend between average sentence length and model performance (correlation coefficient of 0.63, p=0.052).
  • A negative trend was observed between lexical diversity and model performance (correlation coefficient of -0.51, p=0.135), suggesting that higher diversity might lead to better estimation.
  • The analysis of personality traits showed inconclusive but suggestive trends: cooperativeness showed a positive correlation (0.28, p=0.435) and autonomy a positive correlation (0.38, p=0.274), though these were not statistically significant with the limited sample.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Generalizability: The model was trained on data from two companies in specific industries. More data from diverse industries is needed to improve generalizability.
  • Data Bias: The simultaneous collection of texts and happiness scores might have influenced respondents to provide consistent answers. Future studies could explore scenarios where these are collected at different times.
  • Sample Size: The study was conducted with a limited number of participants (10 for validation). Larger cohorts are needed to enhance robustness and applicability.
  • Language Restriction: Experiments were limited to the Japanese language.
  • Nuanced Text Analysis: Free-text descriptions contain rich information beyond happiness. Future work could explore analyzing these texts to understand the determinants of happiness, author experiences, and related emotions more deeply.
  • Individual Characteristics Integration: Incorporating user-specific characteristics like writing style and personality traits more explicitly into the model could improve estimation accuracy, especially for users with shorter or less frequent diary entries.
  • Intervention Strategies: The feasibility of real-time happiness estimation opens avenues for targeted interventions and insights into underlying events or experiences that influence happiness.

Summary updated

BibTeX
@inproceedings{hayashi2024estimation,
  author = {Junko Hayashi and Kazuhiro Ito and Masae Manabe and Yasushi Watanabe and Masataka Nakayama and Yukiko Uchida and Shoko Wakamiya and Eiji Aramaki},
  title = {{Estimation of Happiness Changes through Longitudinal Analysis of Employees’ Texts}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure