Emotion and Sentiment Guided Paraphrasing

Justin Xie, Ameeta AgrawalACL 2023

TL;DR

This paper introduces "fine-grained emotional paraphrasing," a novel task that aims to subtly shift the emotional intensity of text while preserving its core meaning. They developed a framework using pre-trained language models and demonstrated that fine-tuning with emotion labels significantly improves the ability to control emotional transitions in paraphrased text.

What Problem Does This Paper Address?

The internet, particularly social media and online chat, often lacks the non-verbal cues (intonation, facial expressions) that help convey true meaning and emotion. This can lead to misunderstandings, amplified negative sentiments, and even harmful interactions like cyberbullying. While existing paraphrase generation focuses on preserving meaning, and some work addresses emotional style transfer, there's a gap in precisely controlling and subtly adjusting the intensity of emotions within text. This paper tackles the challenge of generating paraphrases that can smoothly transition between different emotional states or simply lower the intensity of an existing emotion, making online communication more nuanced and less prone to extreme emotional reactions.

Methods

The authors propose a framework for fine-grained emotional paraphrasing that involves several key steps:

  1. Dataset Reconstruction: They augmented existing paraphrase datasets (PAWS, MRPC, Quora) by applying a fine-grained emotion classifier (an enhanced GoEmotions model) to label both input and target texts with their emotions and sentiment intensities (neutral, low, high).
  2. Emotion Transition Graph: They conceptualized an "emotion-transition graph" based on emotion gradients, guiding the selection of target emotions. This graph is informed by the GoEmotions dataset's emotion heatmap and sentiment intensity groupings derived from NLTK's Vader scores. The primary goal is to lower emotional intensity.
  3. Prefix Generation: A multi-task approach is used where a prefix is prepended to the input text to guide the language model. This prefix specifies the desired emotion transition (e.g., "anger to annoyance") or sentiment range transition (e.g., "high_neg to low_neg").
  4. Fine-Tuning Language Models: Pre-trained language models (T5, BART, GPT-2) are fine-tuned using these augmented datasets and prefixes under zero-shot, few-shot, and fully supervised settings.

Datasets

The paper utilizes and reconstructs the following publicly available paraphrase datasets:

  • Google PAWS-Wiki (PAWS): Chosen for its translation-based generation and human verification.
  • Microsoft Research Paraphrase Corpus (MRPC): Compiled from news sources.
  • Quora Questions Pairs: Aimed at training "semantic equivalence" models.

These datasets were combined into a "Mix" dataset for overall performance evaluation. The authors note that no dataset is specifically designed for fine-grained emotional paraphrasing, necessitating this reconstruction and augmentation.

Key Insights and Ideas

  • Fine-grained control is achievable: The core idea is that by explicitly guiding language models with emotion transition prefixes, we can achieve more nuanced control over the emotional output of paraphrases than with general style transfer.
  • Emotion intensity matters: The paper highlights that simply changing an emotion isn't enough; controlling its intensity is crucial for moderating online discourse and preventing extreme reactions.
  • Prefixes are powerful guides: The use of task-specific prefixes (e.g., "emotion_A to emotion_B") effectively guides pre-trained models, demonstrating the flexibility of the text-to-text paradigm.
  • Fully supervised and few-shot learning excel: Fine-tuning, even with limited examples (few-shot), significantly outperforms zero-shot approaches in both emotion transition accuracy and paraphrase quality.
  • Sentiment ranges vs. specific emotions: Interestingly, using broader sentiment ranges (high/low positive/negative) as prefixes yielded similar performance to using specific fine-grained emotions, suggesting that the underlying emotion transition patterns are robust.

Key Results

  • Significant performance gains: Fully supervised fine-tuning consistently outperformed zero-shot settings across all models and metrics. Few-shot learning also showed competitive performance.
  • T5 leads in emotion transition: T5 demonstrated superior performance in emotion transition metrics (Exact-SR, Exact-FE) compared to BART and GPT-2, likely due to its multi-task design that accommodates prefixes.
  • BART excels in paraphrasing: BART achieved better scores in traditional paraphrase metrics (BLEU, ROUGE-L, METEOR), indicating higher text similarity and consistency.
  • Few-shot effectiveness: Few-shot learning with 12/3 (train/test) split and above showed substantial improvements over zero-shot.
  • Prefix type similarity: The difference in performance between using sentiment ranges and fine-grained emotions as prefixes was insignificant, suggesting models learn similar transition patterns.

Limitations and Future Work

  • Dataset limitations: The authors acknowledge that existing paraphrase datasets, even when augmented, may not be ideally suited for fine-grained emotional paraphrasing. Developing datasets specifically for this task is a key future direction.
  • Application scope: The current work focuses on lowering sentiment intensity. Future work could explore other application scenarios and emotion transition goals.
  • Emotion classification accuracy: The performance of the GoEmotions classifier, while good, could be further improved, potentially impacting the precision of the emotion transition.
  • Vader score thresholds: The thresholds used for grouping emotions by intensity were empirically chosen and could be refined.
  • Evaluation: While quantitative metrics are used, the authors suggest that well-designed human studies would be beneficial for a more comprehensive evaluation.
  • Model advancements: Future work includes leveraging state-of-the-art large language models (e.g., GPT-4) and exploring more customized model architectures.
  • Ethical considerations: The authors highlight the potential for misuse, such as increasing emotional intensity or enabling censorship, and strongly oppose such applications.

Summary updated

BibTeX
@inproceedings{xie2023emotion,
  author = {Justin Xie and Ameeta Agrawal},
  title = {{Emotion and Sentiment Guided Paraphrasing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure