Standard Named Entity Recognition (NER) pipelines operate under a foundational assumption: the text being processed belongs to a single, predictable language. If a model processes an English sentence, it relies on English syntax, capitalization patterns, and vocabulary distributions to spot a person, location, or organization.
However, in multilingual communities and digital spaces, communication rarely stays within monolingual boundaries. Instead, speakers frequently engage in code-switching—the practice of interweaving multiple languages or dialects within a single conversation or utterance. When languages collide, traditional sequence-labeling models degrade rapidly. To extract information accurately from real-world text, models must decouple entity recognition from rigid monolingual constraints.
The Linguistic Double-Bind of Mixed Text
Why is code-switched NER uniquely difficult? The core challenge stems from an inherent tension between identifying the language of a token and identifying whether it is an entity.
Early foundational work on code-switched data, such as the shared tasks organized by Thamar Solorio et al. (Overview for the First Shared Task on Language Identification in Code-Switched Data) and later by Giovanni Molina et al. (Overview for the Second Shared Task on Language Identification in Code-Switched Data), revealed that named entities are the primary source of error when processing mixed-language text.
Proper nouns frequently bypass the standard phonotactic and grammatical rules of the surrounding language. As Harsh Jhamtani et al. (Word-level Language Identification in Bi-lingual Code-switched Texts) observe, because entities often remain identical across a language switch, they lack distinct linguistic markers. This makes them appear "language-neutral," confusing token-level language identifiers and sequence labelers alike.
Furthermore, entities manifest in highly diverse ways across code-switched text. As categorized by Florian Janke et al. (The University of Texas System Submission for the Code-Switching Workshop Shared Task 2018), an entity can be:
- Language-specific: e.g., "Estados Unidos" vs. "United States".
- Language-neutral but regionally specific: e.g., "Los Angeles".
- Hybrid forms: Blending vocabulary or morphology from both languages.
This complexity is not limited to social media. In technical domains, Yuting Hu et al. (Named Entity Recognition for Chinese biomedical patents) documented a similar phenomenon in Chinese biomedical patents, where specific entities like gene names are written in English while the surrounding context remains in Chinese. Monolingual models fail because they cannot handle these sudden, localized linguistic shifts.
Monolingual NER Assumption:
[ English Context ] -> [ English Entity ] -> [ English Context ]
Code-Switched Reality:
[ Spanish Context ] -> [ Language-Neutral Entity ] -> [ English Context ]
From Joint Labeling to Staged Architectures
Traditional NER models treat entity extraction as a single sequence-labeling task, typically using BIO (Begin, Inside, Outside) tagging with a Conditional Random Field (CRF) or BiLSTM layer. However, in noisy, code-switched environments—where spelling variations and non-standard shorthand run rampant (Parvathy Geetha et al., Tackling Code-Switched NER: Participation of CMU; Devanshu Jain et al., Simple Features for Strong Performance on Named Entity Recognition in Code-Switched Twitter Data)—jointly determining where an entity starts and what semantic class it belongs to becomes overwhelming for a single classifier.
To solve this, modern systems pivot toward two-stage (or staged) architectures. The core intuition is to decouple the structural problem (finding entity boundaries) from the semantic problem (classifying the entity type).
[ Input Text ]
│
▼
┌──────────────────────────┐
│ Stage 1: Span Detection │ ──► Identifies *where* the entity is
└──────────────────────────┘ (Syntactic & local cues)
│
▼
┌──────────────────────────┐
│ Stage 2: Classification │ ──► Identifies *what* the entity is
└──────────────────────────┘ (Semantic & cross-lingual cues)
- Stage 1: Span Boundary Detection. Finding the start and end of a proper noun phrase relies heavily on local syntactic cues rather than global language consistency. Because noun phrase boundaries behave similarly even when languages shift, this stage is highly robust to code-switching.
- Stage 2: Span Classification. Once a span is isolated, it is passed to a classification module. By isolating the entity from the noisy, multi-lingual background context, the classifier can focus purely on semantic categorization.
This staged approach has proven highly successful in competitive benchmarks. Keyu Pu et al. (CMB AI Lab at SemEval-2022 Task 11: A Two-Stage Approach for Complex Named Entity Recognition via Span Boundary Detection and Span Classification) utilized this boundary-then-classification split to handle complex, code-mixed text. Similarly, Mohab Elkaref et al. (NLPeople at SemEval-2023 Task 2: A Staged Approach for Multilingual Named Entity Recognition) demonstrated that separating span extraction from classification significantly mitigates the impact of linguistic noise and low-context settings.
Leveraging Multilingual Representations and Context
Even with a staged architecture, short or highly informal texts (like tweets or search queries) often lack the contextual clues necessary to resolve entity ambiguity. For example, a code-switched token representing a creative work or a title might consist entirely of common words, making it invisible to standard models (Parvathy Geetha et al., Tackling Code-Switched NER: Participation of CMU).
To overcome this, researchers leverage massively multilingual pre-trained language models (like XLM-RoBERTa) combined with three key strategies:
Stacked Embeddings
To capture both fine-grained morphological variations and broad semantic context, models combine multiple tiers of representation. Abdul Aziz et al. (CSECU-DSG at SemEval-2022 Task 11: Identifying the Multilingual Complex Named Entity in Text Using Stacked Embeddings and Transformer based Approach) demonstrated that stacking sub-word transformer representations with character- and word-level embeddings allows models to remain sensitive to internal morphological shifts within a code-switched token while retaining cross-lingual awareness.
Context Augmentation
To combat the brevity of short, code-mixed sentences, the authors of L3i at SemEval-2022 Task 11: Straightforward Additional Context for Multilingual Named Entity Recognition (Emanuela Boros et al.) introduced a method that injects semantically relevant context retrieved from the training data directly into the input. Providing this additional scaffolding helps the transformer resolve ambiguous boundaries that lack local clues.
Self-Training and Ensembling
To maximize generalization on mixed linguistic structures, Abdellah El Mekki et al. (UM6P-CS at SemEval-2022 Task 11: Enhancing Multilingual and Code-Mixed Complex Named Entity Recognition via Pseudo Labels using Multilingual Transformer) integrated a CRF layer on top of a multilingual transformer and used self-training with pseudo-labels to exploit unlabeled multilingual data.
Furthermore, Aapo Pietiläinen et al. (AaltoNLP at SemEval-2022 Task 11: Ensembling Task-adaptive Pretrained Transformers for Multilingual Complex NER) showed that ensembling transformers fine-tuned with diverse random seeds drastically reduces the risk of overfitting to specific language pairs, providing a smoother generalization across mixed linguistic structures.
By moving away from rigid monolingual sequence labeling and embracing staged span detection alongside robust, cross-lingual transformer representations, modern NLP systems can successfully navigate the fluid boundaries of code-switched text.
Key Foundational Work
Word-level Language Identification in Bi-lingual Code-switched Texts (2014) — contributed 1 concept elements to this direction.
Overview for the Second Shared Task on Language Identification in Code-Switched Data (2016) — contributed 1 concept elements to this direction.
Overview for the First Shared Task on Language Identification in Code-Switched Data (2014) — contributed 1 concept elements to this direction.