TL;DR
This paper explores what makes Reddit posts popular, focusing on the linguistic style of posts in Singapore, a diglossic society with both standard English (acrolect) and local Singlish (basilect). The study finds that while standard English appeals to prestige, incorporating Singlish vocabulary helps posts connect with the community, suggesting a "best of both worlds" approach for maximum popularity.
What Problem Does This Paper Address?
Predicting the popularity of online content, particularly on platforms like Reddit, is a significant challenge. Previous research has shown that the style of a post, rather than just its content, is a strong predictor of user engagement (upvotes). However, most of this work has focused on communities with a relatively uniform linguistic style. This paper addresses the gap of understanding how linguistic style influences popularity in a diglossic context, specifically Singapore, where two distinct varieties of English—the formal acrolect (akin to Standard British English) and the informal basilect (Singlish)—coexist. These varieties carry different social connotations: the acrolect is associated with prestige and education, while Singlish is tied to local identity and everyday communication. The paper aims to determine how these competing linguistic appeals affect post popularity on Reddit.
Methods
The authors adopt a three-pronged approach to extract features from Reddit posts, inspired by Bergsma et al. (2012):
- Content Features (BERT): They leverage Bidirectional Encoder Representations from Transformers (BERT), a powerful language model known for capturing nuanced linguistic information, including grammar and semantics. The uncased English BERTBASE model was used to generate post embeddings by averaging sentence-level embeddings.
- Grammatical Features: These were extracted using the spaCy parser, focusing on dependency relations and part-of-speech (POS) tags. The authors also included POS tags of the "heads" of words, which represent the main word in a phrase.
- Style Features: Term frequency-inverse document frequency (TF-IDF) vectors were used, specifically focusing on stopwords and punctuation. This approach aims to capture stylistic elements distinct from core content.
For prediction, the authors employed Random Forest Classifiers, a robust model for imbalanced datasets, weighting each class proportionally to its frequency.
Datasets
The study used data scraped from two Reddit subreddits:
- r/singapore (SG): This subreddit represents the Singaporean context, characterized by diglossia.
- r/unitedkingdom (UK): This subreddit serves as a comparison, representing a more standard English-speaking community.
Data was collected over three years (January 2017 to December 2019) using the Pushshift API. Posts with fewer than 101 characters were excluded to ensure sufficient linguistic content. The popularity of posts was categorized into 8 levels (0-7) based on their score distribution within each subreddit, with Level 0 being the lowest and Level 7 the highest.
Key Insights and Ideas
- Diglossia's Impact: The study highlights that in a diglossic environment, popularity isn't solely driven by prestige (acrolect) or identity (basilect), but a strategic blend of both.
- Prestige vs. Identity: While the acrolect (standard English) appears to confer prestige and attract upvotes, the most successful posts also incorporate elements of the basilect (Singlish) to foster community connection.
- BERT's Power: BERT embeddings proved to be highly effective, often outperforming or matching models that combined grammatical and stylistic features, suggesting its ability to implicitly capture complex linguistic nuances.
- Style Matters: The research reinforces the idea that community-specific styles are crucial for engagement, with posts aligning stylistically to a subreddit's norms being better received.
- Specific Features Shine: More specific stylistic features (like individual punctuation marks) sometimes outperformed broader grammatical features, indicating the importance of fine-grained linguistic cues.
Key Results
- Overall Performance: All models significantly outperformed a simplistic baseline. BERT-based models generally performed best, with only slight improvements when combined with other features.
- SG vs. UK: Models performed slightly better on the UK dataset, possibly due to the more consistent linguistic style compared to Singapore's diglossic situation.
- Level-wise Performance: Unlike some previous studies that found models performed better on lower popularity levels, this paper's models showed better performance at higher popularity levels (Level 7), suggesting that the features used were effective at distinguishing highly popular content.
- Singlish Use: Analysis of Singlish terms showed a U-shaped distribution: the least popular posts used more Singlish (likely due to the basilect's influence), while the most popular posts (Level 7) strategically used Singlish vocabulary alongside standard English to appeal to both identity and prestige.
Limitations and Future Work
- Limitations:
- The study acknowledges that the results are not directly comparable to prior work like Fang et al. (2016) due to differences in data and methodology.
- The authors note that the tools used (like BERT and spaCy) might be biased towards standard English, potentially affecting performance on the Singaporean subreddit.
- Compiling a comprehensive Singlish lexicon was challenging due to variations in romanization and lexical change.
- Future Work:
- The authors suggest incorporating context-free grammar (CFG) features using CoreNLP to further investigate the interplay between Singlish grammatical features and the prevalence of Singlish terms in popular posts.
- This could help confirm or disprove the theory that popular posts use the acrolect's grammatical structure while leveraging the basilect's lexicon.
Summary updated