Detecting Mental Manipulation in Speech via Synthetic Multi-Speaker Dialogue

Run Chen, Wen Liang, Ziwei Gong, Lin Ai, Julia Hirschberg

TL;DR

This paper introduces SPEECHMENTALMANIP, the first benchmark designed to detect mental manipulation in spoken dialogue rather than just text. The authors discover that both state-of-the-art audio-language models and human annotators find identifying manipulative intent significantly more challenging when listening to audio compared to reading transcripts, revealing a critical blind spot in multimodal AI safety.

What Problem Does This Paper Address?

Mental manipulation—the covert use of language to influence or exploit others—is a complex challenge for computational social reasoning. While previous research has developed benchmarks to detect these tactics in text, real-world interactions rely heavily on spoken delivery, including prosody, tone, and timing.

With the rapid rise of audio-capable Large Multimodal Models (LMMs) and the emergence of "voice jailbreaks," understanding how speech affects the expression and detection of manipulation is vital. Currently, there is a lack of controlled, transcript-aligned speech data to isolate how acoustic cues interact with manipulative linguistic strategies. This paper fills that gap by exploring whether spoken delivery makes manipulation easier or harder for AI and humans to detect.

Methods

To systematically isolate the effect of speech modality, the authors developed a two-phase Text-to-Speech (TTS) pipeline to convert text dialogues into high-quality, multi-speaker audio:

  1. Turn-Level Audio Generation: Dialogue transcripts are parsed, and individual speakers are deterministically assigned a distinct voice from a curated pool of six ElevenLabs voices (varying in gender, age, and accent) to maintain identity across conversational turns.
  2. Conversation Reconstruction: Individual turn utterances are normalized for loudness and concatenated with a controlled 0.2-second silence between turns to preserve natural conversational rhythm.

The authors evaluated Qwen2.5-Omni-7B (an advanced audio-language model) using a 4-shot batch evaluation pipeline. The model was tasked with making a binary decision (YES/NO) on the presence of manipulation via constrained decoding, followed by predicting the specific manipulation tactic and extracting a supporting quote.

Additionally, the authors conducted a human re-annotation study with eight fluent English speakers who independently evaluated 100 conversations across both text-only and audio-only modalities to measure inter-annotator agreement.

Datasets

The study introduces SPEECHMENTALMANIP, which is built on top of the consensus split (MENTALMANIP_CON) of the existing text-based MENTALMANIP dataset (originally derived from the Cornell Movie Dialogues corpus).

The evaluation set used in this paper consists of transcript-aligned audio clips, including:

  • 609 manipulative audio clips
  • 90 non-manipulative audio clips

Key Insights and Ideas

  • The Sensitivity-Specificity Gap: Audio-language models exhibit high specificity but very low recall. They are conservative judges; they excel at avoiding false alarms on safe audio but miss the vast majority of actual manipulative speech.
  • Acoustic Tactic Collapse: When evaluating audio, the model collapses its predictions toward a few acoustically salient categories. It heavily over-relies on Intimidation (characterized by harsh acoustic pressure/loudness) and Persuasion or Seduction (characterized by warmth), while completely missing more subtle, semantically driven tactics like Rationalization, Denial, or Evasion.
  • Modality-Induced Ambiguity: Spoken delivery actually introduces layers of subjective interpretation that make the task harder for humans too. Human inter-annotator agreement dropped significantly when evaluating the audio modality compared to text, proving that prosody can obscure or complicate perceived intent.
Figure 1: An example dialogue from the SPEECHMEN- TALMANIP dataset. The Qwen2.5 model is given the audio (transcript shown for clarity), but fails to detect manipulation.

Key Results

  • Model Performance: Qwen2.5-Omni-7B achieved an accuracy of 82.2% on non-manipulative audio (GT=NO) but plummeted to just 34.8% accuracy on manipulative audio (GT=YES).
  • Tactic Distribution: For true positives, the model's tactic predictions were heavily skewed: 49.4% were labeled as Intimidation and 29.9% as Persuasion or Seduction, ignoring the diverse distribution of the ground-truth text labels.
  • Human Agreement: Human majority votes aligned with the original dataset labels at a rate of 0.72 for text, but this dropped to 0.56 for audio. The high-agreement human group saw their inter-annotator consistency (Krippendorff’s Alpha) slide from 0.526 on text down to 0.422 on audio.
Figure 3: Pair-wise Cohen’s Kappa between Human Annotators for Text modality
Figure 4: Pair-wise Cohen’s Kappa between Human Annotators for Audio modality

Limitations and Future Work

The authors note several limitations in their current setup:

  • Synthetic Speech Constraints: The synthetic dialogues are generated turn-by-turn, meaning they lack natural conversational features like overlapping speech, interruptions, or backchanneling.
  • Model and Prompt Scope: The evaluation relied on a single model architecture (Qwen2.5-Omni) and a standard few-shot prompt that did not include explicit definitions of the manipulation tactics.

Future Directions: The authors plan to expand the benchmark using more diverse, natural human speech and explore definition-augmented prompting strategies to help models better navigate the inherent subjectivity and multimodal ambiguity of conversational manipulation.

Summary updated

Abstract

Mental manipulation, the strategic use of language to covertly influence or exploit others, is a newly emerging task in computational social reasoning. Prior work has focused exclusively on textual conversations, overlooking how manipulative tactics manifest in speech. We present the first study of mental manipulation detection in spoken dialogues, introducing a synthetic multi-speaker benchmark SPEECHMENTALMANIP that augments a text-based dataset with high-quality, voice-consistent Text-to-Speech rendered audio. Using few-shot large audio-language models and human annotation, we evaluate how modality affects detection accuracy and perception. Our results reveal that models exhibit high specificity but markedly lower recall on speech compared to text, suggesting sensitivity to missing acoustic or prosodic cues in training. Human raters show similar uncertainty in the audio setting, underscoring the inherent ambiguity of manipulative speech. Together, these findings highlight the need for modality-aware evaluation and safety alignment in multimodal dialogue systems.
BibTeX
@inproceedings{chen2026detecting,
  author = {Run Chen and Wen Liang and Ziwei Gong and Lin Ai and Julia Hirschberg},
  title = {{Detecting Mental Manipulation in Speech via Synthetic Multi-Speaker Dialogue}},
  booktitle = {IWSDS},
  year = {2026},
}
Figure