Abstract
This study presents a comprehensive, long-term project to explore the effectiveness of various prompting techniques in detecting dialogical mental manipulation. We implement Chain-of-Thought prompting with Zero-Shot and Few-Shot settings on a binary mental manipulation detection task, building upon existing work conducted with Zero-Shot and Few-Shot prompting. Our primary objective is to decipher why certain prompting techniques display superior performance, so as to craft a novel framework tailored for detection of mental manipulation. Preliminary findings suggest that advanced prompting techniques may not be suitable for more complex models, if they are not trained through example-based learning.
Introduction
Detecting subtle mental manipulation in conversation is a significant challenge for AI because it lacks the obvious markers of toxic speech. Mental manipulation is a nuanced form of psychological influence that is often context-dependent and subtle. Unlike hate speech or blatant harassment, manipulative language doesn't always rely on "toxic" keywords, making it difficult for standard Natural Language Processing (NLP) classifiers to identify.
This research investigates the "complexity paradox": does a more powerful model automatically improve at detecting social nuance, or does its increased complexity require more structured guidance? We specifically explore whether advanced prompting techniques—specifically those that encourage Large Language Models (LLMs) to articulate their reasoning process—can bridge the gap in identifying these subtle social interactions.
Methodology
The researchers compared four distinct prompting strategies across two models, GPT-3.5 and GPT-4o, using the MentalManipCon dataset (Wang et al., 2024), which contains over 4,000 human-annotated dialogues validated by consensus.
- Zero-Shot: The model identifies manipulation without prior examples.
- Few-Shot: The model is provided with three labeled examples (two manipulative, one non-manipulative) to establish a baseline.
- CoT (Zero-Shot): The model is prompted to "think step-by-step" to reach a conclusion but is given no example reasoning paths.
- CoT (Few-Shot): Native English speakers manually annotated 42 examples with detailed, step-by-step reasoning to "teach" the model how to evaluate a dialogue.
Results
The effectiveness of the models was measured primarily by Accuracy and F1-scores. As shown in the table below, the Few-Shot CoT setting achieved the highest accuracy for both models (0.722 for GPT-3.5 and 0.778 for GPT-4o).
A significant finding was the failure of Zero-Shot CoT for the more advanced model. In this setting, GPT-4o’s Macro F1 score was the lowest among all techniques. Both models saw a reduction in Precision when using Zero-Shot CoT, indicating a high rate of false positives—labeling normal conversations as manipulative.
Discussion and Insights
The "Over-Thinking" Trap
One of the most surprising findings was that GPT-4o performed poorly in the Zero-Shot CoT setting. When asked to reason without examples, the more "intelligent" model began over-analyzing benign traits. It frequently flagged informal speech or fragmented sentences as signs of manipulation, essentially "hallucinating" malicious intent in communication styles.
Examples are Mandatory for Complex Models
While GPT-3.5 was relatively stable, GPT-4o’s performance swung wildly based on the prompt. This suggests that as models become more complex, they require example-based learning (Few-Shot) to ground their reasoning, especially for subjective tasks like social interaction analysis. The models often struggled to distinguish between a "confrontational communication style" and "actual manipulative intent."
Limitations and Future Work
The study's reliance on 42 manually annotated reasoning chains for Few-Shot CoT suggests that expanding this dataset could further stabilize performance. Future work will explore alternative prompting frameworks like Iterative Prompting or Tree-of-Thoughts and investigate how demographic biases and the "serial position effect" impact detection accuracy.
Summary updated