Pouring Your Heart Out: Investigating the Role of Figurative Language in Online Expressions of Empathy

Gyeongeun Lee, Christina Wong, Meghan Guo, Natalie PardeACL 2024 · DOI

Introduction

Empathy serves as the cornerstone of online support communities, yet teaching machines to recognize it remains a significant challenge. Traditional empathy detection often relies on "black-box" models or generalized datasets that fail to capture the nuances of supportive communication. Researchers from the University of Illinois at Chicago have identified a critical missing link: figurative language. Expressions such as "pouring one's heart out" or being "in the same boat" are non-literal but carry profound emotional resonance. This study investigates whether explicitly identifying idioms, metaphors, and hyperboles can enhance AI's ability to detect empathy within domain-specific contexts, specifically online forums for acne sufferers.

Methodology

The research team developed a two-stage framework to integrate figurative language into empathy classification.

  1. Figurative Language Detection: Using a multi-task framework based on the mT5 model, the researchers scanned posts for three types of figurative speech: idioms (e.g., "keep your chin up"), metaphors (e.g., "acne is a curse"), and hyperboles (e.g., "my mood changes every second").
  2. Empathy Classification: The team evaluated both traditional feature-based models (SVM, Logistic Regression with LIWC) and pre-trained language models (RoBERTa-large, T5).
  3. FIG Integration: The core innovation involved appending detected figurative language labels—termed FIG—as additional features to these models to measure performance gains.

The AcnEmpathize Dataset

The study utilized AcnEmpathize, a custom dataset of 12,212 individual posts from acne.org. Manual annotation revealed that approximately 24% of posts were empathetic. Notably, figurative language was found to be the norm rather than the exception, appearing in 61% of all posts. As shown in the distribution below, empathetic posts are significantly more likely to contain idioms and metaphors compared to non-empathetic ones.

Distribution of figurative language types across empathetic and non-empathetic posts.

Key Insights: The Language of Empathy

Statistical analysis confirmed that the way users express themselves is deeply tied to the presence of empathy.

  • Significance of Idioms and Metaphors: Chi-square tests demonstrated that idioms and metaphors are highly significant indicators of empathy. In contrast, while hyperbole is common, it does not statistically differentiate empathetic from non-empathetic responses.
  • The "War" Metaphor: Metaphors in this community often revolve around "fighting" or "battling" acne, framing the condition as a shared struggle.
  • Emotional Variance: Metaphors were associated with higher intensities of "fear" and "anger," whereas idioms often conveyed optimism (e.g., "light at the end of the tunnel").
Statistical significance of figurative language categories in empathy detection.

Results and Performance

Integrating figurative language features led to substantial improvements in model performance:

  • State-of-the-Art Results: The RoBERTa-twitter-sentiment + FIG model achieved the highest F1 score of 0.809, compared to 0.789 without figurative features.
  • Impact on Simple Models: The benefit was most pronounced for traditional classifiers. An SVM model that failed to identify any empathetic posts (0.000 F1) saw its performance jump to 0.488 F1 after the inclusion of FIG features.
  • Precision Gains: The models became significantly more reliable, showing a marked increase in precision when identifying empathetic content.

Conclusion

This research proves that "how" a message is delivered is as vital as its literal content for detecting empathy. While the study faced challenges like label imbalance and the inherent subjectivity of empathy, it paves the way for more emotionally intelligent AI. Future work may expand to include other linguistic devices such as sarcasm and similes to further refine the machine's understanding of human support.

Summary updated

BibTeX
@inproceedings{lee2024pouring,
  author = {Gyeongeun Lee and Christina Wong and Meghan Guo and Natalie Parde},
  title = {{Pouring Your Heart Out: Investigating the Role of Figurative Language in Online Expressions of Empathy}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.31},
}
Figure