Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching

Kalika Bali, Pushpak Bhattacharyya, Marina Fomicheva, Philipp Koehn, Holger Schwenk, Alan W. Black, Mona Diab, Shuguang Chen, Sunayana Sitaram, Víctor Soto, Anirudh Srinivasan, Gustavo Aguilar, Elena Álvarez Mellado, Amitava Das 0001, Indranil Dutta, Alexander Gelbukh, Sudipta Kar, Manuel Mager, Parth Patwa, Yihong Theis, Van Dung Pham, Dama Sravani, Lalitha Kameswari, Radhika Mamidi, Vivek Srivastava, Mayank Singh, Ruthanna Barnett, Eva Codó, Eva Duran Eppler, Montserrat Forcadell, Pénélope Gardner-Chloros, Roeland van Hout, Melissa G. Moyer, Maria Carme Torras, María Teresa, Devansh Gautam, Kshitij Gupta, Manish Shrivastava, Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer, Myle Ott, Jingfei Du, M. Joshi, Danqi Chen, Omer Levy, Veselin Stoyanov, Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Suraj Pandey, Suman Dowlagar, Jatin Sharma, Monojit Choudhury, Yogarshi Vyas, Scott Baldauf, Somnath Banerjee, Sudip Kumar Naskar, Paolo Rosso, Sivaji Bandyopadhyay, Utsab Barman, Amitava Das, Joachim Wagner, Jennifer Foster, Fahad Alghamdi, Julia Hirschberg, Thamar Solorio, Kunal Chakma, Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, Leonardo NevesOpenalex Unknown 2021 · DOI · OpenAlex

Introduction

Code-switching (CSW) is a linguistic phenomenon where multilingual speakers alternate between two or more languages within a single conversation or text. While prevalent in daily speech and increasingly common on social media, current Natural Language Processing (NLP) technology—including large pretrained models like mBERT and BART—often struggles to process these mixed-language inputs effectively. The Fifth Workshop on Computational Approaches to Linguistic Code-Switching (CALCS-2021) brought together researchers to address these gaps, focusing on resource development, methodological innovation, and the unique challenges of machine translation in CSW environments.

The Challenge of Code-Switching in Social Media

CSW is most frequently observed in casual settings. As digital communication becomes more informal, social media platforms have become the primary repositories for code-switched data. However, this data presents significant hurdles: it is often scarce, lacks standard orthography (especially in transliterated languages like Hinglish), and involves complex grammatical interleaving that standard monolingual models cannot easily parse.

Social media platforms like Facebook are primary sources for code-switched data.

Methodological Innovations

To overcome the lack of high-quality parallel corpora, workshop contributors explored several creative strategies:

  • Synthetic Data Generation: Researchers utilized lexical substitution and back-translation to create large-scale artificial datasets, providing a necessary foundation for training.
  • Curriculum Learning: Top-performing systems employed a two-stage process: initial training on synthetic data followed by "polishing" on smaller, high-quality human-annotated sets.
  • The "English Pivot" Strategy: Several papers demonstrated that translating code-switched text into a high-resource language like English before performing downstream tasks (like sentiment analysis) often outperforms direct classification on the original mixed text.
  • Hierarchical Meta-Embeddings (HME): To address out-of-vocabulary issues, models combined word, sub-word, and character-level representations to improve robustness in tasks like Named Entity Recognition (NER).

Shared Tasks and Benchmarks

A central component of CALCS-2021 was a shared task on Machine Translation (MT) in CSW settings, featuring both supervised and unsupervised modalities:

  • Supervised: English to Hinglish (Hindi-English).
  • Unsupervised: Spanish-English (Spanglish) to English, and Modern Standard Arabic-Egyptian Arabic (MSA-EA) to English.

Beyond the shared task, the proceedings introduced diverse benchmarks, including a Telugu political speech corpus, the JOSA sentiment dataset for Jopara (Guarani-Spanish), and Bengali-English datasets for detecting abusive content in transliterated social media posts.

Key Insights and Results

The workshop yielded several critical findings for the field:

  1. Metrics vs. Perception: Research indicated that standard metrics like the Code-Mixing Index (CMI) do not always correlate with human readability or semantic coherence. A sentence can be linguistically complex but practically unintelligible.
  2. Model Limitations: Even models trained on 100+ languages (e.g., mBERT) require explicit fine-tuning to understand the specific syntactic patterns of how languages interleave in CSW.
  3. Performance Milestones: The top English-Hinglish MT system achieved a 12.67 BLEU score using mT5 and curriculum learning. In sentiment analysis for Jopara, the Spanish-specific BETO model (macro-accuracy 0.64) surprisingly outperformed broader multilingual models.

Future Outlook

While CALCS-2021 highlights significant progress, several hurdles remain. Future research must address multi-sentence coherence where the "matrix" language shifts, improve inference speeds for real-time applications like voice assistants, and move beyond basic tagging toward complex reasoning and question-answering in code-switched environments. The introduction of the CodemixedNLP toolkit marks a vital step toward standardizing these specialized methods for the broader research community.

Summary updated

Abstract

Bienvenidos to the proceedings of the fifth edition of the workshop on computational approaches for linguistic code-switching (CALCS-2021)!Code-switching is this very interesting phenomenon where multilingual speakers communicate by moving back and forth between the languages they speak when communicating with other multilingual speakers.Code-switching (CSW) is predominantly used in speech but since it also tends to be more prevalent in casual settings, we can observe CSW in genres like social media platforms where interactions tend to be more casual.However interesting, our current NLP technology is lagging behind in the development of resources and methodologies that can effectively process code-switched language.This is true for even the large multilingual pretrained models such as mBERT and BART.At the same time, the growing adoption of smart devices and automated assistants that rely on speech interfaces, makes it even more pressing that our field addresses CSW language data.This workshop series brings together experts and practitioners that are currently working on different aspects of CSW with a special focus on motivating tighter collaborations between speech and text researchers.We received 18 regular workshop submissions, of which we accepted 13.But this year we also had a special submission type called "Rising Stars".The goal of the Rising Stars is to allow young scientists that have recently published work in the space of CSW to present this work again to a specialized audience.These submissions are non-archival and are intended to increase visibility of CSW research by young researchers.We received two submissions of this type and we hope to continue this new track in future editions.Our workshop also aims to motivate new research and energize the community to take on the challenges posed by CSW data.With this in mind, we hosted a new shared task on machine translation in CSW settings colocated with the workshop.This shared task provided two modalities for participation, supervised and unsupervised.For the supervised mode we asked participants to translate English data into Hinglish (Hindi-English).For the unsupervised setting we provided the following language pairs: Spanish-English (Spanglish) to English, English to Spanglish, Modern Standard Arabic-Egyptian Arabic (MSA-EA) to English and English to MSA-EA.The current leaderboard for the task shows 12 individual public system submissions coming out of 5 different teams.The overview of the shared task and the individual system submissions will be presented at the workshop.The workshop program includes short talks from regular workshop submissions, rising star talks and system description talks.We also have a stellar invited speaker program with talks by Özlem Çetinoglu, Manish Shrivastava and Ngoc Thang Vu.In addition, the one day program will also feature an exciting panel discussing research challenges unique to Machine Translation in CSW environments.
BibTeX
@inproceedings{bali2021proceedings,
  author = {Kalika Bali and Pushpak Bhattacharyya and Marina Fomicheva and Philipp Koehn and Holger Schwenk and Alan W. Black and Mona Diab and Shuguang Chen and Sunayana Sitaram and Víctor Soto and Anirudh Srinivasan and Gustavo Aguilar and Elena Álvarez Mellado and Amitava Das and Indranil Dutta and Alexander Gelbukh and Sudipta Kar and Manuel Mager and Parth Patwa and Yihong Theis and Van Dung Pham and Dama Sravani and Lalitha Kameswari and Radhika Mamidi and Vivek Srivastava and Mayank Singh and Ruthanna Barnett and Eva Codó and Eva Duran Eppler and Montserrat Forcadell and Pénélope Gardner-Chloros and Roeland van Hout and Melissa G. Moyer and Maria Carme Torras and María Teresa and Devansh Gautam and Kshitij Gupta and Manish Shrivastava and Yinhan Liu and Jiatao Gu and Naman Goyal and Xian Li and Sergey Edunov and Marjan Ghazvininejad and Mike Lewis and Luke Zettlemoyer and Myle Ott and Jingfei Du and M. Joshi and Danqi Chen and Omer Levy and Veselin Stoyanov and Adam Paszke and Sam Gross and Francisco Massa and Adam Lerer and James Bradbury and Gregory Chanan and Trevor Killeen and Zeming Lin and Natalia Gimelshein and Luca Antiga and Alban Desmaison and Andreas Köpf and Edward Z. Yang and Zachary DeVito and Martin Raison and Suraj Pandey and Suman Dowlagar and Suman Dowlagar and Kalika Bali and Jatin Sharma and Monojit Choudhury and Yogarshi Vyas and Scott Baldauf and Somnath Banerjee and Sudip Kumar Naskar and Paolo Rosso and Sivaji Bandyopadhyay and Utsab Barman and Amitava Das and Joachim Wagner and Jennifer Foster and Gustavo Aguilar and Fahad Alghamdi and Victor Soto and Mona Diab and Julia Hirschberg and Thamar Solorio and Somnath Banerjee and Kunal Chakma and Sudip Kumar Naskar and Amitava Das and Paolo Rosso and Sivaji Bandyopadhyay and Monojit Choudhury and Francesco Barbieri and Jose Camacho-Collados and Luis Espinosa Anke and Leonardo Neves and Alexander F. Gelbukh},
  title = {{Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching}},
  booktitle = {Openalex Unknown},
  year = {2021},
  doi = {10.18653/v1/2021.calcs-1},
}
Figure