Dealing with Controversy: An Emotion and Coping Strategy Corpus Based on Role Playing

Enrica Troiano, Sofie Labat, Marco Stranisci, Rossana Damiano, Viviana Patti, Roman KlingerEMNLP 2024 · DOI

Abstract

Traditional Natural Language Processing (NLP) research often treats emotions as static labels. However, psychological theories suggest that emotions trigger dynamic behavioral responses known as coping strategies. This paper introduces COPING, a novel dataset designed to identify how individuals respond to conflict. Using a role-playing methodology, the researchers captured four distinct strategies—Attack, Contact, Distance, and Reject—across controversial topics. Results indicate that while these strategies are identifiable in text and behavioral descriptions, they present a significant challenge for current state-of-the-art language models.

Introduction

Most NLP research focuses on sentiment or emotion classification (e.g., "angry" vs. "sad"). Yet, in real-world interactions, internal emotional states lead to external "action tendencies." For instance, two individuals experiencing anger may react differently: one might Attack the source of frustration, while another might Distance themselves to avoid escalation. Current computational models lack the structured data necessary to bridge the gap between internal feelings and external coping behaviors. This study addresses this gap by providing a corpus that links textual dialogue and behavioral descriptions to specific psychological coping mechanisms.

The COPING Framework

The authors developed a multi-stage framework to elicit high-quality, labeled data while minimizing researcher bias.

The multi-stage role-playing framework for data elicitation, involving persona definition, scenario generation, and role-playing.

As illustrated in the framework above, the process involves:

  1. Persona Definition: Translating Roseman’s (2013) psychological theory into four character profiles (X).
  2. Scenario Generation: Utilizing GPT-4 to create 60 provocative scenarios across five sensitive topics (Abortion, Immigration, Racism, LGBTQ+, and Drugs).
  3. Data Elicitation: Crowdsourced participants "impersonate" character X by writing a verbal reply to an interlocutor (Y), describing X’s non-verbal behavior, and rating X’s emotions.
  4. Validation: Pre-tests ensured that character descriptions were distinct enough for human recognition before the main data collection phase.

The COPING Corpus

The resulting COPING (COPIng emo tioN strateGies) corpus consists of 1,200 balanced data points. Each entry includes a dialogue reply, a description of non-verbal behavior (e.g., "X clenches their fists"), and fine-grained emotion ratings. The dataset is publicly available to facilitate research into behavioral intent in social contexts.

Analysis and Results

Benchmarking with Small Language Models (SLMs) like DistilBERT and RoBERTa, as well as Large Language Models (LLMs) like Dolphin-2, revealed several key insights:

  • Behavioral Cues vs. Dialogue: Models generally performed better when analyzing behavioral descriptions (.54–.55 F1) than when analyzing dialogue alone (.52 F1). This suggests that verbal language is often more "masked" or polite, whereas physical descriptions more clearly signal intent.
  • The "Contact" Advantage: The "Contact" strategy—characterized by attempts to understand or unite—was the easiest to identify. This strategy often employs clear linguistic markers such as "I understand."
Emotional profile of the "Contact" coping strategy, showing a distribution across various emotional states.
  • Difficulty of Disengagement: Strategies involving disengagement, such as "Distance" and "Reject," were significantly harder for both humans and models to detect. The authors suggest that role-playing "moving away" from a stimulus is cognitively more difficult to perform in a writing task.
  • Performance Benchmarks: Human performance reflected the subjectivity of the task, with an inter-annotator agreement of Fleiss’ kappa = .34. Model performance was comparable, with the best F1 scores ranging from .52 to .55. Performance varied significantly by topic, with Abortion reaching .63 F1 while Immigration dropped to .36 F1.

Conclusion and Future Work

The COPING corpus demonstrates that psychological coping strategies can be successfully elicited through role-playing and identified in text, though they remain a difficult target for current NLP models. While the dataset's scale and Western cultural focus are limitations, it provides a foundation for developing virtual agents capable of responding to user behavioral intent rather than just surface-level sentiment. Future work will explore these patterns in spontaneous social media data and across diverse cultural contexts.

Summary updated

BibTeX
@inproceedings{troiano2024dealing,
  author = {Enrica Troiano and Sofie Labat and Marco Stranisci and Rossana Damiano and Viviana Patti and Roman Klinger},
  title = {{Dealing with Controversy: An Emotion and Coping Strategy Corpus Based on Role Playing}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.89},
}
Figure