TL;DR
This paper introduces a computational method to detect "perspective-getting" – the active effort to understand another's viewpoint – in online discussions. By training a RoBERTa model on annotated Wikipedia conversations, they demonstrate that this behavior is associated with more constructive dialogue, particularly when it initiates a discussion.
What Problem Does This Paper Address?
Effective interpersonal understanding relies on grasping what others think and feel. While "perspective-taking" (imagining another's viewpoint) is well-studied, "perspective-getting" (actively seeking this information) is a more direct strategy that has been under-explored computationally. The authors aim to bridge this gap by developing a method to automatically detect perspective-getting in large-scale online discussions, specifically on Wikipedia, and to understand its impact on discussion dynamics.
Methods
The researchers developed a codebook to operationalize perspective-getting into two categories:
- Attending the Other's Perspective (AOP): This involves quoting, summarizing, or referencing parts of the other's message to show understanding or engagement.
- Asking for Clarification (AFC): This category focuses on directly asking questions to understand the other's thoughts, feelings, or beliefs.
They then annotated a dataset of Wikipedia discussions using this codebook. A RoBERTa model was fine-tuned on this annotated data to classify messages for AOP and AFC, achieving a macro F1 score of 0.76. They also explored logistic regression as a baseline.
Datasets
The primary dataset used is WikiDisputes, derived from Wikipedia talk pages. This dataset contains 133,019 messages from non-escalated discussions and 4,441 messages from escalated discussions. The authors pre-processed this dataset by removing long messages, duplicates, and incomplete discussions. A subset of these messages was annotated for perspective-getting.
Key Insights and Ideas
- Perspective-getting is more than just empathy: While related to understanding others, perspective-getting is framed as an active, observable strategy (asking, attending) rather than a purely cognitive state (imagining).
- Wikipedia as a rich environment: The platform's nature, where editors must reach consensus, makes it an ideal setting to study how understanding others influences discussion outcomes.
- The power of initiation: Discussions that start with a message demonstrating AOP are more likely to see subsequent messages also exhibiting AOP, suggesting a positive feedback loop.
- AFC is harder to detect: While AOP showed a stronger association with discussion outcomes, AFC proved more challenging for both human annotators and the model, indicating nuances in its expression.
Key Results
- Model Performance: The fine-tuned RoBERTa model achieved a macro average F1 score of 0.76 for detecting AOP and AFC. RoBERTa outperformed a logistic regression baseline, particularly for AFC.
- AOP vs. AFC Prevalence: In the annotated dataset, AOP was present in 85% of messages, while AFC was present in only 25%.
- Escalation: Contrary to their hypothesis, perspective-getting (both AOP and AFC) was not significantly higher in non-escalated discussions compared to escalated ones. This might be due to Wikipedia's inherent goal of reaching agreement and the influence of other factors like politeness.
- Discussion Course (H2): Discussions starting with a message showing AOP were more likely to have subsequent messages also showing AOP. This effect was significant for AOP but not for AFC. This suggests that initial perspective-getting can foster more perspective-getting throughout the discussion.
Limitations and Future Work
- Annotation Challenges: The binary nature of the categories might oversimplify nuanced expressions of perspective-getting. The visibility of external links in the dataset was also a limitation for annotating control variables.
- Limited Data: The number of annotated messages, especially for escalated discussions, was relatively small, potentially impacting the generalizability of findings related to escalation.
- Domain Specificity: The study is limited to English Wikipedia discussions. The generalizability to other domains (e.g., social media discussions about politics) where the goal might not be consensus needs further investigation.
- Future Work: The authors suggest exploring more fine-grained categories of perspective-getting, investigating its role in online polarization, and applying the developed classifiers to other domains. They also note that future models could benefit from considering the entire conversation context, not just the reply message.
Summary updated