Does Social Pressure Drive Persuasion in Online Fora?

Ayush Jain, Shashank SrivastavaEMNLP 2021 · OpenReview

Abstract

Online forums such as ChangeMyView have been explored to research aspects of persuasion and argumentative quality in language. While previous research has focused on arguments between a view-holder and a persuader, we explore the premise that apart from the merits of arguments, persuasion is influenced by the ambient social community. We hypothesize that comments from the rest of the community can either affirm the original view or implicitly exert pressure to change it. We develop a structured model to capture the ambient community’s sentiment towards the discussion and its effect on persuasion. Our experiments show that social features themselves are significantly predictive of persuasion (even without looking at the actual content of discussion), with performance comparable to some earlier approaches that use content features. Combining community and content features leads to overall performance of 78.5% on the persuasion prediction task. Our analyses suggest that the effect of social pressure is comparable to the difference between persuasive and non-persuasive language strategies in driving persuasion and that social pressure might be a causal factor for persuasion.

Introduction: Beyond Dyadic Interaction

Most computational research on persuasion focuses on the "dyadic" interaction: the exchange between a view-holder (the Original Poster or OP) and a persuader (the Root Replier or RR). While the quality of an argument is vital, humans are social creatures susceptible to peer pressure. This research investigates a gap in current literature: the role of the ambient social community.

Figure 1: A visualization of the discussion path involving the Original Poster (OP), Root Replier (RR), and the ambient community.

In a forum like Reddit's r/ChangeMyView, dozens of other users chime in, affirming the original view or implicitly pressuring the OP to reconsider. This paper asks whether we can predict persuasion by looking at these "third-party" comments and how the community's collective sentiment evolves over the course of a discussion.

Methodology: Modeling the Temporal Dynamics of Persuasion

The researchers developed a structured model designed to capture the temporal dynamics of a conversation. The model extracts the specific sequence of interactions between the OP and the RR leading to a "Delta" (the symbol used in CMV to signify a change of heart).

At each step in the conversation, the model maintains a "social vector" representing the community's state. This vector includes global sentiment (a running average of sentiment intensity using VADER) and support/against counts (a tally of community members supporting or opposing either side).

Figure 2: The LSTM architecture processes content embeddings alongside learned social vectors to predict the likelihood of persuasion.

The architecture utilizes a Long Short-Term Memory (LSTM) network. At each time step, the LSTM processes the content of the current OP/RR comment (represented via BERT or GloVe embeddings) alongside the updated social vector. In the most advanced variant, a secondary LSTM "learns" the social representation directly from the community comments' embeddings, rather than relying solely on hand-engineered sentiment features.

Dataset and Experimental Setup

The study utilizes data from the ChangeMyView (CMV) subreddit. To ensure a robust "community" presence, the researchers filtered for discussions with at least 50 comments. Pairs of successful persuasion attempts and unsuccessful ones were collected, ensuring comparable discussion path lengths. The final dataset included roughly 10,400 training instances and 2,400 test instances, with a balanced split between positive and negative cases.

Key Results and Insights

The experiments compared several model variants on the task of predicting persuasion:

  • Social Features Only: 58.43% Accuracy.
  • Content Only (BERT): 75.93% Accuracy.
  • BERT + Social (Hand-engineered): 78.09% Accuracy.
  • BERT + Social LSTM (Learned): 78.51% Accuracy.

One of the most striking findings is that social features alone—without looking at a single word written by the OP or the persuader—can predict persuasion with 58.4% accuracy. This is significantly better than a random guess and comparable to some early linguistic-only models. Furthermore, structured models (LSTMs) significantly outperformed unstructured ones (like Logistic Regression), proving that the evolution of community sentiment over time carries more information than just the final aggregate sentiment.

Discussion: Social Pressure as a Catalyst

Through counterfactual analysis, the authors found that "adding" supportive community comments to a failed discussion in the model often flipped the prediction to "persuaded." Temporal analysis further supports a potential causal link: in 65% of successful cases, the Delta was awarded after at least half of the community comments had already appeared. This suggests the community isn't just reacting to the OP's change of mind; they are likely driving it.

While the findings are rooted in the specific culture of r/ChangeMyView, they highlight that social influence is a critical factor in changing opinions online. Future work may involve developing more complex neural architectures to model the multi-party nature of these discussions more deeply and exploring how social pressure manifests in less structured online environments.

Summary updated

Abstract

Online forums such as ChangeMyView have been explored to research aspects of persuasion and argumentative quality in language. While previous research has focused on arguments between a view-holder and a persuader, we explore the premise that apart from the merits of arguments, persuasion is influenced by the ambient social community. We hypothesize that comments from the rest of the community can either affirm the original view or implicitly exert pressure to change it. We develop a structured model to capture the ambient community’s sentiment towards the discussion and its effect on persuasion. Our experiments show that social features themselves are significantly predictive of persuasion (even without looking at the actual content of discussion), with performance comparable to some earlier approaches that use content features. Combining community and content features leads to overall performance of 78.5% on the persuasion prediction task. Our analyses suggest that the effect of social pressure is comparable to the difference between persuasive and non-persuasive language strategies in driving persuasion and that social pressure might be a causal factor for persuasion.
BibTeX
@inproceedings{jain2021does,
  author = {Ayush Jain and Shashank Srivastava},
  title = {{Does Social Pressure Drive Persuasion in Online Fora?}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=RKyNdQMtyV},
}
Figure