Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics

Jiarui Liu, Yueqi Song, Yunze Xiao, Mingqian Zheng, Lindia Tjuatja, Jana Schaich Borg, Mona Diab, Maarten Sap

Introduction

As Large Language Models (LLMs) are increasingly integrated into sensitive domains such as education, healthcare, and customer service, developers often "steer" these models using specific personas to align with brand voices or user demographics. However, the extent to which these assigned identities—encompassing demographics, political ideologies, and personality traits—alter the underlying moral logic of the model remains poorly understood. This research investigates how multi-dimensional personas influence LLM moral reasoning and their ability to persuade others during simulated "Socratic debates."

The Challenge of Intersectional Moral Reasoning

Most prior research into AI ethics has focused on "toy" problems, such as the trolley problem, or isolated single traits like gender or age. This study addresses a critical gap by exploring intersectional identities. By examining how a 20-year-old libertarian from China might reason differently than a 60-year-old conservative from the United States, the researchers provide a more nuanced view of how complex identities clash and converge in dynamic, multi-turn moral deliberations.

Methodology

The researchers developed a structured framework for Synthetic Socratic Debates to observe these dynamics in action:

  1. Persona Modeling: A six-dimensional persona space was defined, including Age (20–60), Gender (M/F/Non-binary), Country (6 options), Social Class (3 options), Political Ideology (4 quadrants), and Big Five Personality traits (High/Low).
  2. Debate Simulation: Two agents with conflicting initial stances on a moral dilemma were paired to engage in up to five turns of debate, moderated by a third AI instance.
  3. Evaluation Framework: The study utilized Moral Foundation Theory (MFT) to measure values like Care, Fairness, Authority, Loyalty, Sanctity, and Liberty. It also analyzed Aristotle’s Modes of Persuasion—Ethos (credibility), Pathos (emotion), and Logos (logic).
  4. Models Tested: The experiments utilized GPT-4o, Claude-3.5-Sonnet, LLaMA-4-Maverick, and Qwen3-235B-A22B.

Dataset Selection

The study utilized the SCRUPLES ANECDOTES corpus, specifically 131 interpersonal dilemmas from the "Am I the Asshole?" (AITA) subreddit. These scenarios were selected for their high level of controversy, where human annotators originally showed significant disagreement, providing a fertile ground for testing nuanced moral reasoning.

Key Insights and Results

The Dominance of Ideology and Personality

The research reveals that political ideology and personality traits are the primary predictors of how an LLM will judge a moral situation, significantly outweighing demographic factors like age or gender.

Consensus and Persuasion Dynamics

  • The Liberal Advantage: Personas assigned as "Libertarian-Left" or "High Openness" were significantly more successful at reaching consensus. These agents tended to use more empathetic, emotional appeals (Pathos).
  • The Authoritarian Wall: Authoritarian personas (both Left and Right) were the least efficient debaters. They frequently failed to reach agreement and relied heavily on appeals to authority (Ethos).
  • Persuasive Ability: Interestingly, Libertarian-Right personas demonstrated the highest persuasive ability, achieving high consensus rates while maintaining their own "self-alignment" (staying true to their initial stance).

Argumentative Evolution

As debates progressed, the agents' strategies shifted. While their confidence (measured by log-probabilities) increased, their use of emotional and credibility-based appeals decreased. Over time, the agents moved toward more "tempered," logic-heavy (Logos) argumentation.

Psychological Mirroring

LLMs remarkably mirror human psychological data. For example, older personas were found to be more "deontological" (strict and rule-based), while female personas prioritized "Care" foundations, aligning with established findings in human moral psychology.

Limitations and Future Work

While the study provides a large-scale look at persona effects, it is limited by its reliance on English-language dilemmas from a Western-centric platform (Reddit). The authors acknowledge that language and cultural context heavily influence persuasive behavior. Furthermore, the debates were limited to dyadic (two-agent) interactions, whereas real-world moral deliberation often involves complex group dynamics.

Future research directions include exploring multi-party dialogues and developing "persona-aware" evaluation frameworks. Such frameworks are essential to ensure AI systems remain transparent about how their assigned identities influence the advice and moral judgments they provide to users.

Summary updated

BibTeX
@misc{liu2025synthetic,
  author = {Jiarui Liu and Yueqi Song and Yunze Xiao and Mingqian Zheng and Lindia Tjuatja and Jana Schaich Borg and Mona Diab and Maarten Sap},
  title = {{Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics}},
  year = {2025},
  eprint = {2506.12657},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.12657},
}
Figure