Abstract
While much NLP research focuses on detecting online toxicity, this paper explores the "bright side" of social media by identifying "connective language"—text that signals intellectual humility and a willingness to engage with opposing views. The authors compare a fine-tuned BERT model with a few-shot GPT-3.5 Turbo classifier using a novel dataset of 14,107 posts from Facebook, Twitter, and Reddit. Findings indicate that BERT significantly outperforms GPT-3.5 (F1 = 0.85 vs. 0.48), particularly in recall. Furthermore, correlation analysis demonstrates that connective language is a unique construct distinct from mere politeness, offering a new metric for fostering healthy online deliberation.
Introduction
Most social media NLP research addresses the "dark side" of the internet, focusing on toxicity, misinformation, and bot detection. While vital for content moderation, this focus neglects the "bright side"—language that fosters healthy, pro-democratic deliberation. This study addresses the lack of tools for identifying connective language, defined as text expressing intellectual humility, openness to other perspectives, and a genuine willingness to engage with ideologically unaligned individuals. Identifying such content is a critical step toward building platforms that recommend thoughtful exchanges.
Methodology
The researchers compared two distinct machine learning approaches to classify connectivity:
- Fine-tuned BERT: Utilizing
bert-base-uncased, the model was specialized for the task through fine-tuning on a balanced sample of human-labeled data. - Few-shot GPT-3.5 Turbo: This approach used OpenAI’s model via API, provided with a detailed prompt containing definitions and examples of connectivity.
The BERT classification pipeline, from initial human labeling to final prediction, is visualized below.
Dataset and Validation
The authors constructed a novel multi-platform dataset of 14,107 social media posts from Reddit, Twitter (X), and Facebook (2012–2022). To capture rare connective instances, they targeted specific subreddits (e.g., r/ChangeMyView) and used keyword queries like "I hear you" or "see where you're coming from."
Human labeling was conducted by four coders (1 = connective, 0 = non-connective), achieving a reliable Krippendorff’s alpha of 0.73. An online survey of 621 users further validated that public perception of "connective" posts aligns with the researchers' definitions.
Key Results
The fine-tuned BERT model significantly outperformed GPT-3.5 Turbo across all metrics. While GPT-3.5 suffered from low recall (0.42)—missing over half of the connective posts—BERT achieved an overall F1-score of 0.85.
Performance also varied by platform. BERT performed exceptionally well on Twitter (F1 = 0.98) but found Reddit more challenging (F1 = 0.76), likely due to the latter's longer and more complex discourse.
Insights: Connectivity vs. Politeness
A critical finding is that connectivity is a distinct construct. While it correlates slightly with politeness (~0.28), a post can be connective without being traditionally "polite" (e.g., using informal language while acknowledging an opponent's point). Furthermore, connectivity was negatively correlated with toxicity and incivility, suggesting it is actively pro-social rather than just "not toxic."
Conclusion and Limitations
Specialized, smaller models like BERT still hold a major edge over general-purpose LLMs for nuanced social-science tasks. However, the study notes limitations, including sampling bias toward connective language and a focus on English-language text. Future work should explore newer models like GPT-4o and expand to audio/video-centric platforms like TikTok or YouTube.
Summary updated