Do Stochastic Parrots have Feelings Too? Improving Neural Detection of Synthetic Text via Emotion Recognition

Alan Cowap, Yvette Graham, Jennifer Foster

Abstract

Generative AI models, often termed "stochastic parrots," lack the emotional drivers inherent in human writing, leading to a phenomenon called "affective incoherence." This study proposes improving synthetic text detection by first training models to recognize human emotions. The resulting framework, emoPLMsynth, significantly enhances detection accuracy and outperforms existing state-of-the-art detectors—and even ChatGPT itself—in identifying machine-generated content.

Introduction

As Large Language Models (LLMs) become more sophisticated, the "arms race" between text generation and detection is accelerating. Most current detection methods focus on statistical artifacts, such as word frequency distributions or "perplexity." However, as models improve and sampling strategies evolve, these statistical signatures become increasingly difficult to distinguish from human writing.

The authors argue that current detection methods miss a crucial "human factor": emotion. Humans write with intent, driven by feelings that color word choices and narrative flow. AI models, by contrast, exhibit an "affective deficit." While they may use emotional vocabulary, they lack an emotional driver, leading to "affective incoherence"—where the emotional tone is inconsistent or lacks the logical progression found in human-authored content.

Methodology

The researchers propose a novel transfer learning pipeline called emoPLMsynth. The process operates in three distinct stages to bridge the gap between general language understanding and emotional awareness:

  1. Emotion Pre-training (emoPLM): A standard Pretrained Language Model (such as BERT or BLOOM) is fine-tuned on an emotion classification task using news headlines labeled with Ekman’s six basic emotions (Joy, Sadness, Fear, Anger, Disgust, Surprise).
  2. Task Adaptation: The multi-class emotion classification head is removed and replaced with a binary classification head (Human vs. Synthetic).
  3. Detection Fine-tuning (emoPLMsynth): The "emotionally aware" model is fine-tuned on the specific task of synthetic text detection.
The emoPLMsynth pipeline: transitioning from standard PLM to emotion-aware detection.

Datasets and Benchmarks

The study utilized several key datasets, including two new benchmarks developed by the authors:

  • NEWSsynth: A dataset of 40,000 news articles, split evenly between human-authored (RealNews) and synthetic (Grover-generated) content.
  • ChatGPT100: A high-quality test set of 100 articles across five domains (Science, Sport, Business, etc.), consisting of 50 human articles and 50 generated by ChatGPT-3.5.
  • GoodNewsEveryone: Used for initial emotion training, containing 5,000 news headlines annotated with categorical emotions.

Key Insights

  • The Affective Deficit: AI makes "emotional errors" rather than just grammatical ones. It lacks the "emotional coherence" that acts as a glue for human storytelling.
  • Emotion as a Robust Signal: Unlike statistical methods like perplexity, which struggle when AI models use "nucleus sampling" to mimic human randomness, emotional patterns remain a more robust differentiator for provenance detection.
  • ChatGPT’s Self-Blindness: Strikingly, ChatGPT is poor at recognizing its own output in a zero-shot setting, frequently misclassifying its own synthetic text as human.

Results

The emotionally aware detector (emoBERTsynth) achieved an accuracy of 76.23% on the RealNews-Test set, significantly outperforming the baseline BERT (67.20%) and larger models like the 124M parameter GPT-2 detector.

Comparison of performance metrics between the baseline BERTsynth and the emotion-aware emoBERTsynth.

As shown in the performance metrics, the emotional fine-tuning helped the model achieve a superior balance between precision and recall. On the ChatGPT100 dataset, emoBERTsynth reached an accuracy of 76.00%, while ChatGPT itself only managed 60.00% when asked to identify its own writing.

Limitations and Future Work

While promising, the authors acknowledge several limitations. The study focused exclusively on English-language models, and emotional expression varies significantly across cultures. Additionally, the models were limited to a 512-token input; longer documents might provide more "emotional rope" for detection but require higher computational costs. Future research will investigate "affective profiles" to see if specific AI models have unique emotional "fingerprints" that make them easier to track.

Summary updated

Abstract

Recent developments in generative AI have shone a spotlight on high-performance synthetic text generation technologies. The now wide availability and ease of use of such models highlights the urgent need to provide equally powerful technologies capable of identifying synthetic text. With this in mind, we draw inspiration from psychological studies which suggest that people can be driven by emotion and encode emotion in the text they compose. We hypothesize that pretrained language models (PLMs) have an affective deficit because they lack such an emotional driver when generating text and consequently may generate synthetic text which has affective incoherence i.e. lacking the kind of emotional coherence present in human-authored text. We subsequently develop an emotionally aware detector by fine-tuning a PLM on emotion. Experiment results indicate that our emotionally-aware detector achieves improvements across a range of synthetic text generators, various sized models, datasets, and domains. Finally, we compare our emotionally-aware synthetic text detector to ChatGPT in the task of identification of its own output and show substantial gains, reinforcing the potential of emotion as a signal to identify synthetic text. Code, models, and datasets are available at https: //github.com/alanagiasi/emoPLMsynth
BibTeX
@inproceedings{cowap2023do,
  author = {Alan Cowap and Yvette Graham and Jennifer Foster},
  title = {{Do Stochastic Parrots have Feelings Too? Improving Neural Detection of Synthetic Text via Emotion Recognition}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.665},
}
Figure