Abstract
In contexts where debate and deliberation is the norm, participants are regularly presented with new information that conflicts with their original beliefs. When required to update their beliefs (belief alignment), they may choose arguments that align with their worldview (confirmation bias). We test this and competing hypotheses in a constraint-based modeling approach to predict the winning arguments in multi-party interactions in the Reddit ChangeMyView dataset. We impose structural constraints that reflect competing hypotheses on a hierarchical generative Variational Auto-encoder. Our findings suggest that when arguments are further from the initial belief state of the target, they are more likely to succeed.
Introduction
In the digital arena of Reddit’s "ChangeMyView," winning arguments aren't the ones that play it safe. When individuals encounter information contradicting their beliefs, they typically either update their views (belief alignment) or reinforce their existing stance (confirmation bias). This study seeks to determine which paradigm dominates in real-world online debates by quantifying "belief" and "distance" through latent state modeling. By analyzing multi-party interactions, the researchers aim to predict which arguments successfully persuade an "Opinion Holder" to change their mind.
Methodology
The authors employ a hierarchical generative Variational Auto-encoder (VAE) to model the latent belief states ($Z$) of both the Opinion Holder ($O$) and the Commenter ($C$). The core innovation is the use of distance-based structural modeling constraints. Instead of allowing the neural network to learn unguided, specific geometric hypotheses are imposed on the latent space to evaluate which psychological theory best predicts a "winning" argument (marked by a $\Delta$ in the dataset).
These hypotheses, ranging from confirmation bias ($h_2$) to the "alternate hypothesis" ($h_1$), are operationalized as loss functions using ranking and contrastive margins. This forces the model to organize its latent representations according to these specific theories.
Experimental Setup
The study utilizes the Reddit ChangeMyView (CMV) dataset, split into two primary evaluation settings:
- In-Domain (ID): Training and testing on the same high-level topics (e.g., Sports).
- Cross-Domain (CD): Training on one set of topics and testing on entirely different ones to evaluate the model's ability to generalize.
To eliminate length-based bias, sentences were truncated to 100 tokens, and comments under five words were removed.
Key Results
The researchers benchmarked their constrained model against standard baselines and state-of-the-art encoders. The most striking finding is that winning arguments tend to be mathematically "farther" from the target's initial belief in the latent space, suggesting that users in deliberative environments are moved by novel information rather than echoes of their own thoughts.
As shown in the results table, the $h_4$ hypothesis—which posits that successful arguments are not irrelevant but are "far" from the original opinion—achieved an AUC of 69.7 in cross-domain settings. This represented a statistically significant improvement ($p < 0.05$) over the null hypothesis ($h_0$). Furthermore, this theory-driven approach achieved results competitive with much larger models, such as Pre-trained BERT Sentence Encoders (AUC 69.7), despite the VAE's more constrained architecture.
Discussion and Future Work
This research demonstrates that classical psychological theories, like cognitive dissonance and belief alignment, can serve as effective structural constraints to improve the interpretability and performance of generative models. However, the authors note that these findings are specific to "ChangeMyView," a community designed for open-mindedness. Future work will explore whether these dynamics hold in "echo chamber" environments or face-to-face interactions where non-verbal cues and social pressure may alter the "distance" required for persuasion. Additionally, the framework may be applied to study the believability of misinformation based on its distance from a target's belief state.
Summary updated