Disentangling Codemixing in Chats: The NUS ABC Codemixed Corpus

Svetlana Churina, Akshat Gupta, Nur Insyirah Binte Imam Mujtahid, Kokil Jaidka

TL;DR

Researchers from the National University of Singapore have introduced the ABC (Air Batu Campur) Codemixed Corpus, a massive dataset of over 355,000 private instant messages. Unlike previous datasets scraped from public social media, this corpus is author-labeled with relationship metadata, allowing researchers to study how "code-mixing"—switching between languages like English, Mandarin, and Malay—functions as a tool for managing intimacy and social dynamics in private digital spaces.

What Problem Does This Paper Address?

Most Natural Language Processing (NLP) research on code-mixing relies on "performative" data from platforms like Twitter or Reddit. However, the way we speak in public is vastly different from how we chat with our best friends or colleagues on WhatsApp.

The authors identify a major gap: we lack high-quality, large-scale corpora of private conversations that include both the linguistic data and the social context (like how close the speakers are). Without this context, it is difficult to build models that truly understand the nuances of multilingual communication, such as why a speaker might switch to a specific dialect to soften a request or express frustration.

Methods

The researchers employed a "voluntary data donation" framework, where 166 participants contributed 477 chat threads from platforms like Telegram and WhatsApp.

To process this "messy" real-world data, they developed a sophisticated pipeline:

  • Preprocessing: Standard cleaning (removing PII, URLs, and timestamps) while preserving emojis and casing to maintain linguistic signals.
  • Language Identification (LID): Instead of relying solely on standard models like fastText, which struggle with informal slang, they used Qwen2.5-14B-Instruct for token-level tagging. This allowed them to identify "Singlish" (Singaporean English) particles and transliterated words that traditional models often miss.
  • Translation: They used the same LLM to translate entire conversations into English, finding that conversation-level context was necessary because message-level translation often failed due to a lack of context.
  • Three-Level Analysis: They analyzed the data through statistical patterns, linguistic features (using LIWC-22), and social functions (using the Systemic Functional Linguistics framework).

Datasets

  • The NUS ABC Codemixed Corpus (New): 355,641 messages spanning English, Mandarin, Hokkien, Malay, and Tamil. It includes metadata on relationship type (Close, General, Work) and self-reported intimacy scores.
  • Baselines for Evaluation: The authors compared their LLM-based processing against fastText, XLM-RoBERTa, Google Translate, and Facebook’s NLLB-200.

Key Insights and Ideas

  • Intimacy Drives Mixing: Code-mixing is most frequent in short messages and close relationships. In intimate conversations, the mixing rate for short messages (0–5 tokens) hits 36.7%, compared to significantly lower rates in professional exchanges.
  • LLMs as Annotators: The study demonstrates that LLMs like Qwen2.5 are superior to traditional LID models for "low-resource" informal varieties. They are better at identifying irregular English (slang/abbreviations) and transliterated Asian languages.
  • The "Tenor" of Chat: Using Systemic Functional Linguistics, the authors categorized code-mixing into three "tenors":
  • Affective: Using markers like sayang (Malay for "dear") or jialat (Hokkien for "troublesome") to manage emotions.
  • Interpersonal-Social: Using markers like paiseh (Hokkien for "embarrassed/sorry") to maintain politeness and rapport.
  • Discourse-Stylistic: Using particles like ah, hor, meh to manage the flow of conversation and signal group identity.

Key Results

  • Corpus Scale: The final dataset contains 355,641 messages with an average length of 4.3 tokens. Roughly 22% of all messages are code-mixed.
  • LID Performance: The LLM-based tagging showed "superior performance" over fastText and XLM-RoBERTa, particularly for proper nouns and transliterated words.
  • Translation Nuance: When comparing LLM translations to Google Translate, the LLM achieved high BERT scores (0.87), indicating strong semantic similarity, despite low BLEU scores. This suggests the LLM is paraphrasing to capture the meaning and tone of the code-mixed text rather than just doing word-for-word replacement.

Limitations and Future Work

The authors acknowledge several constraints:

  • Demographic Bias: The data comes primarily from Singaporean university students (aged 18–34), so the findings might not generalize to older adults or different professional contexts.
  • Excerpt-Based Data: Participants donated selected snippets rather than full histories to protect privacy, which limits the ability to analyze long-term turn-taking or shifts in tone over time.
  • Intentionality: Linguistic analysis can capture associations, but it cannot definitively prove the intent behind every language switch without follow-up interviews.

Future work involves expanding the corpus to include more diverse demographics and exploring dialect-specific adaptation for NLP models.

Summary updated

BibTeX
@inproceedings{churina2026disentangling,
  author = {Svetlana Churina and Akshat Gupta and Nur Insyirah Binte Imam Mujtahid and Kokil Jaidka},
  title = {{Disentangling Codemixing in Chats: The NUS ABC Codemixed Corpus}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.80},
}
Figure