TL;DR
This paper introduces SPEECHMENTALMANIP, the first benchmark designed to detect mental manipulation in spoken dialogue rather than just text. The authors discover that both state-of-the-art audio-language models and human annotators find identifying manipulative intent significantly more challenging when listening to audio compared to reading transcripts, revealing a critical blind spot in multimodal AI safety.
What Problem Does This Paper Address?
Mental manipulation—the covert use of language to influence or exploit others—is a complex challenge for computational social reasoning. While previous research has developed benchmarks to detect these tactics in text, real-world interactions rely heavily on spoken delivery, including prosody, tone, and timing.
With the rapid rise of audio-capable Large Multimodal Models (LMMs) and the emergence of "voice jailbreaks," understanding how speech affects the expression and detection of manipulation is vital. Currently, there is a lack of controlled, transcript-aligned speech data to isolate how acoustic cues interact with manipulative linguistic strategies. This paper fills that gap by exploring whether spoken delivery makes manipulation easier or harder for AI and humans to detect.
Methods
To systematically isolate the effect of speech modality, the authors developed a two-phase Text-to-Speech (TTS) pipeline to convert text dialogues into high-quality, multi-speaker audio:
- Turn-Level Audio Generation: Dialogue transcripts are parsed, and individual speakers are deterministically assigned a distinct voice from a curated pool of six ElevenLabs voices (varying in gender, age, and accent) to maintain identity across conversational turns.
- Conversation Reconstruction: Individual turn utterances are normalized for loudness and concatenated with a controlled 0.2-second silence between turns to preserve natural conversational rhythm.
The authors evaluated Qwen2.5-Omni-7B (an advanced audio-language model) using a 4-shot batch evaluation pipeline. The model was tasked with making a binary decision (YES/NO) on the presence of manipulation via constrained decoding, followed by predicting the specific manipulation tactic and extracting a supporting quote.
Additionally, the authors conducted a human re-annotation study with eight fluent English speakers who independently evaluated 100 conversations across both text-only and audio-only modalities to measure inter-annotator agreement.
Datasets
The study introduces SPEECHMENTALMANIP, which is built on top of the consensus split (MENTALMANIP_CON) of the existing text-based MENTALMANIP dataset (originally derived from the Cornell Movie Dialogues corpus).
The evaluation set used in this paper consists of transcript-aligned audio clips, including:
- 609 manipulative audio clips
- 90 non-manipulative audio clips
Key Insights and Ideas
- The Sensitivity-Specificity Gap: Audio-language models exhibit high specificity but very low recall. They are conservative judges; they excel at avoiding false alarms on safe audio but miss the vast majority of actual manipulative speech.
- Acoustic Tactic Collapse: When evaluating audio, the model collapses its predictions toward a few acoustically salient categories. It heavily over-relies on Intimidation (characterized by harsh acoustic pressure/loudness) and Persuasion or Seduction (characterized by warmth), while completely missing more subtle, semantically driven tactics like Rationalization, Denial, or Evasion.
- Modality-Induced Ambiguity: Spoken delivery actually introduces layers of subjective interpretation that make the task harder for humans too. Human inter-annotator agreement dropped significantly when evaluating the audio modality compared to text, proving that prosody can obscure or complicate perceived intent.
Key Results
- Model Performance: Qwen2.5-Omni-7B achieved an accuracy of 82.2% on non-manipulative audio (GT=NO) but plummeted to just 34.8% accuracy on manipulative audio (GT=YES).
- Tactic Distribution: For true positives, the model's tactic predictions were heavily skewed: 49.4% were labeled as Intimidation and 29.9% as Persuasion or Seduction, ignoring the diverse distribution of the ground-truth text labels.
- Human Agreement: Human majority votes aligned with the original dataset labels at a rate of 0.72 for text, but this dropped to 0.56 for audio. The high-agreement human group saw their inter-annotator consistency (Krippendorff’s Alpha) slide from 0.526 on text down to 0.422 on audio.
Limitations and Future Work
The authors note several limitations in their current setup:
- Synthetic Speech Constraints: The synthetic dialogues are generated turn-by-turn, meaning they lack natural conversational features like overlapping speech, interruptions, or backchanneling.
- Model and Prompt Scope: The evaluation relied on a single model architecture (Qwen2.5-Omni) and a standard few-shot prompt that did not include explicit definitions of the manipulation tactics.
Future Directions: The authors plan to expand the benchmark using more diverse, natural human speech and explore definition-augmented prompting strategies to help models better navigate the inherent subjectivity and multimodal ambiguity of conversational manipulation.
Summary updated