Detecting Code-Switching between Turkish-English Language Pair

Zeynep Yirmibeşoğlu, Gülşen EryiğitEMNLP 2018

TL;DR

This paper tackles the challenge of automatically detecting code-switching between Turkish and English in social media text, a phenomenon that complicates natural language processing. They introduce the first dedicated Turkish-English code-switching dataset and achieve a strong 95.6% micro-averaged F1-score using Conditional Random Fields (CRFs) with character n-grams and lexicon features.

What Problem Does This Paper Address?

Code-switching, the practice of alternating between languages within a single conversation or text, is increasingly prevalent, especially in social media. This linguistic fluidity poses a significant hurdle for natural language processing (NLP) tasks, as standard tools often struggle to handle the mix of languages. Existing NLP approaches, particularly those focused on text normalization, can misinterpret foreign words as errors and attempt to correct them, hindering accurate analysis. While code-switching has been studied in other language pairs, there was a notable lack of research and resources specifically for the Turkish-English pair, particularly within the noisy domain of social media.

Methods

The authors employ a two-pronged approach:

  • Character N-gram Language Modeling: They utilize character n-grams (sequences of characters) to model the language of Turkish and English. This method is less sensitive to individual word errors and can capture sub-word patterns. They experiment with different training corpora, including formal and social media data.
  • Conditional Random Fields (CRFs): CRFs are a powerful statistical modeling technique for sequence labeling tasks. In this context, they are used to label each word in a sentence as either Turkish or English. The CRF model incorporates several features:
  • Lexicon Lookup (LEX): Boolean features indicating the presence of a word in Turkish or English dictionaries.
  • Character N-gram Language Model (LM): Features derived from the output of their character n-gram models.
  • Apostrophe Feature (APOS): A novel feature designed to handle Turkish's agglutinative nature. Turkish often uses apostrophes to separate suffixes from proper nouns. The authors hypothesize that code-switchers might follow a similar pattern with foreign words. This feature checks for apostrophes and analyzes the sub-part before it, leveraging language dictionaries.

Datasets

  • Monolingual Corpora:
    • ETD (English Training Data): 10M English tokens from news resources (formal language).
  • TNC (Turkish National Corpus): Primarily formal Turkish words.
  • TTC (Turkish Twitter Corpus): 6M Turkish tokens extracted from Twitter, representing informal, user-generated content.
  • Code-Switched Test Corpus (CSC): A newly introduced dataset consisting of 391 social media posts (from Twitter and Ekşi Sözlük) containing Turkish-English code-switching. This corpus comprises 5430 tokens, with approximately 30% being English words. It was cross-annotated by two human annotators.

Key Insights and Ideas

  • Social Media Data Matters: The character n-gram models trained on informal social media data (TTC) performed significantly better than those trained on formal data (TNC). This highlights the importance of using domain-specific data for code-switching detection.
  • Apostrophes as Clues: The introduction of the APOS feature, which leverages the apostrophe usage in Turkish, proved beneficial. This feature helps the model identify potential intra-word code-switching, a common pattern when foreign words are integrated into Turkish sentences.
  • CRFs Outperform N-grams: The CRF models consistently outperformed the character n-gram models, demonstrating their effectiveness in sequence labeling for this task.
  • Morphological Differences Aid Detection: The authors note that the morphological distinctiveness between Turkish and English, along with their different character sets, aids in language identification.

Key Results

The paper reports a 95.6% micro-averaged F1-score for their best performing model, which is a CRF model incorporating lexicon, LM, and APOS features. This represents a significant achievement for Turkish-English code-switching detection. For comparison, a simple baseline assigning "Turkish" to all tokens achieved only a 72.6% accuracy and 42.1% macro-average F1.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that their code-switched test dataset is relatively small (391 posts).
  • English Performance: While overall performance is high, they note that there is still room for improvement in correctly identifying English tokens, as indicated by the relatively lower F1 scores for English.
  • Future Directions: The authors plan to increase the corpus size and explore other sequence modeling techniques, such as LSTMs, to further enhance performance.

Summary updated

BibTeX
@inproceedings{yirmibesoglu2018detecting,
  author = {Zeynep Yirmibeşoğlu and Gülşen Eryiğit},
  title = {{Detecting Code-Switching between Turkish-English Language Pair}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure