TL;DR
This paper introduces CodeSwitch-Reddit, a large, novel dataset of written multilingual discourse from online forums, and uses it to explore how findings from oral code-switching (CS) translate to written contexts. They discover that written CS on Reddit exhibits distinct topical preferences and subtle differences in informality and linguistic proficiency compared to monolingual posts, suggesting CS is a complex strategy beyond simply bridging lexical gaps.
What Problem Does This Paper Address?
While oral code-switching (CS) has been extensively studied for decades, its written counterpart, particularly in online environments, remains significantly under-researched. This is largely due to a lack of large-scale, diverse datasets that capture the nuances of written multilingual communication. Existing research on written CS often relies on limited data, making it difficult to draw broad conclusions or compare findings with oral CS. This paper aims to bridge this gap by creating a substantial dataset and using it to investigate whether established sociolinguistic patterns in oral CS—related to content, style, and speaker proficiency—hold true in the context of online discussion forums.
Methods
The authors introduce the CodeSwitch-Reddit corpus, a new dataset of written code-switched messages. To build this corpus, they first identified country-specific subreddits likely to host multilingual content and extracted millions of posts. They then applied a series of filters to isolate true code-switching, excluding simple replies, named entities, quotes, and explicit translations. A probabilistic tool called polyglot was used for initial language identification. To ensure the quality of the identified CS posts, they conducted a rigorous manual annotation process with three annotators per post, achieving high agreement.
For their analysis, they employed:
- Topic Modeling (MALLET): To compare the topical preferences of code-switched posts against monolingual posts from the same authors and subreddits.
- Informality Marker Analysis: Using a formal-informal parallel dataset (GYAFC) to identify markers of informal style and then measuring their frequency in CS versus monolingual posts.
- Linguistic Competence Metrics: Calculating various lexical and grammatical features (e.g., type-to-token ratio, lexical density, mean age of acquisition, sentence length) to compare authors who frequently code-switch with those who do not.
Datasets
- CodeSwitch-Reddit Corpus: A newly introduced dataset comprising over 135,000 code-switched messages from five language pairs (English-Tagalog, English-Greek, English-Romanian, English-Indonesian, English-Russian) across multiple bilingual communities on Reddit. An additional dataset with lower precision was also collected.
- Monolingual Reddit Dataset: A comparison dataset of English monolingual posts from the same subreddits and authors as the CodeSwitch-Reddit corpus.
- GYAFC (Formal-Informal Parallel Dataset): Used to extract stylistic markers indicative of informal language.
- English Wikipedia Corpus: Used for word ranking to filter out less common English words.
- External Lexical Resources: For calculating mean age of acquisition and word concreteness.
Key Insights and Ideas
- Written CS has its own "flavor": The study reveals that code-switched posts on Reddit tend to focus on different topics than monolingual posts, particularly emphasizing relationships and personal experiences, suggesting CS can delineate social and emotional territory.
- CS is not just about lexical gaps: While some studies suggest oral CS is used to bridge vocabulary deficits, the findings here suggest that mixing languages in writing might also involve advanced grammatical capabilities to maintain utterance coherence.
- Subtle informality in written CS: Even within the generally informal environment of Reddit, code-switched posts show a statistically significant, albeit small, increase in informality markers compared to monolingual posts from the same authors.
- Proficiency is nuanced: High code-switchers exhibit slightly lower lexical sophistication but higher grammatical complexity, challenging the notion that CS is solely a marker of lower linguistic competence.
Key Results
- Topical Differences: Topic modeling identified distinct topical distributions. Code-switched posts showed a higher prevalence of topics related to relationships and family, while monolingual posts more frequently discussed politics and issues.
- Informality: The mean frequency of informality markers in CS posts was 0.160, compared to 0.153 in monolingual posts from the same authors. A Wilcoxon signed-rank test confirmed this difference was statistically significant (p < 1.01e-23).
- Linguistic Competence: High code-switchers showed statistically significant lower proficiency in lexical metrics (e.g., NTTR, mean AoA) but higher proficiency in some grammatical metrics (e.g., sentence length, parse tree depth) compared to low code-switchers. These differences were small but reliably detected.
Limitations and Future Work
- Language Pairs: The current corpus primarily focuses on English as a common language, limiting the exploration of CS between other language pairs.
- Annotation Subjectivity: While efforts were made to ensure precision, some subjective interpretations of what constitutes "true" code-switching might still exist, particularly with shared lexical items.
- Future Work: The authors plan to extend the dataset with conversational context and investigate additional aspects of online written multilingual discourse. They also aim to break down the data by individual native languages to capture differences in proficiency stemming from various L1 backgrounds.
Summary updated