Persuasion of the Undecided: Language vs. the Listener

Liane Longpre, Esin Durmus, Claire CardieArgMining @ ACL 2019 · DOI

Abstract

This study investigates how persuasion strategies differ when targeting undecided versus decided audience members in online debates. By analyzing data from debate.org, researchers found that undecided voters prioritize content-rich, semantic language, whereas those who flip their stance from an opposing side are more influenced by stylistic cues and rhetorical framing. The results demonstrate that audience characteristics, such as prior beliefs, significantly interact with linguistic features to determine persuasive outcomes, suggesting that the "secret sauce" of persuasion depends entirely on the listener's initial state.

Introduction

In Natural Language Processing (NLP), research on persuasion has traditionally focused on the inherent strength of the message. However, persuasion does not occur in a vacuum; a message that convinces a neutral observer might fail to move someone with a deeply held opposing view. This paper addresses a critical gap by distinguishing between persuading the undecided (moving from neutral to a side) and persuading the decided (flipping from one side to the other). While psychology and political science suggest these groups react differently to information, computational models have rarely treated them as distinct populations.

Methodology

The researchers framed persuasion as a binary classification task: predicting which side (PRO or CON) convinced a voter given a specific debate. They employed Logistic Regression and performed extensive feature ablation to identify the most influential factors. The study specifically compared two distinct audience transitions:

  • FROM-MIDDLE: Voters who were undecided before the debate and chose a winner after.
  • FROM-OPPOSING: Voters who started on one side and "flipped" to the other.
Visualizing the two persuasion scenarios: FROM-MIDDLE (undecided to decided) and FROM-OPPOSING (flipping sides).

Data and Feature Engineering

The study utilized a dataset from debate.org, comprising 67,315 debates and over 36,000 users. The researchers categorized features into two main buckets:

  1. Linguistic Features: Lexical items (TF-IDF, modal verbs), style (length, citations, direct address), and semantics (sentiment, subjectivity, politeness).
  2. Audience Features: Derived from user profiles, including demographics (gender), "matching ideology" (shared political/religious labels), and "opinion similarity."

Opinion similarity was calculated using a unique aspect of the dataset: the "Big Issues" section of user profiles. By analyzing stances on topics like abortion or the death penalty, the researchers built high-dimensional "belief vectors" for each listener to quantify their worldview.

The process of encoding user profiles from debate.org into a "Decidedness encoding" based on their stances on "The BIG Issues."

Key Insights and Results

The models significantly outperformed majority baselines, achieving 69.01% accuracy for undecided voters and 67.22% for those flipping stances. Several key patterns emerged:

  • Substance vs. Style: Undecided voters are more influenced by content-centric features like subjectivity and modal verbs (e.g., "should," "must"). Conversely, voters who flip their stance are more sensitive to stylistic markers, such as the use of citations and direct engagement with the opponent.
  • The Echo Chamber Effect: For undecided voters, "opinion similarity" was a significantly stronger predictor of persuasion than for those flipping sides. Sharing a general worldview makes an undecided voter much easier to convince.
  • Subjectivity as a Double-Edged Sword: Removing subjectivity features improved model accuracy for the "FROM-OPPOSING" group. This suggests that for hostile audiences, emotional or subjective language may be counterproductive, whereas it helps guide the undecided.
  • Credibility Markers: "Flippers" appear to look for formal markers of credibility. Features like citations and referring to the opponent were crucial for predicting flips, but less relevant for the undecided group.

Conclusion

This research highlights that the effectiveness of persuasive language is contingent upon the audience's prior beliefs. While linguistic features are generally more powerful than audience data alone, the interaction between the two is vital. Future work may explore more complex neural architectures to capture subtle rhetorical patterns and investigate how these dynamics shift in different debate domains beyond politics and religion.

Summary updated

BibTeX
@inproceedings{longpre2019persuasion,
  author = {Liane Longpre and Esin Durmus and Claire Cardie},
  title = {{Persuasion of the Undecided: Language vs. the Listener}},
  booktitle = {ArgMining @ ACL},
  year = {2019},
  doi = {10.18653/v1/w19-4519},
}
Figure