Code-switched text mixes two or more languages in one sentence, as in "vamos a ver Orange is the New Black tonight." Recognizing named entities in such text (people, places, products, book and film titles) is hard. The strongest systems do not rely on a multilingual encoder alone. They look entities up in external knowledge, either lists of known names or Wikipedia pages. They also always look up in the same way, whichever language the entity or its surroundings happen to be in.
We propose to treat the lookup as a routing decision. For each candidate entity span, the system chooses which language-specific knowledge source to consult, or whether to consult one at all. The choice is conditioned on the local language mix around the span, and it is made under a budget on the number of lookups. The design borrows from link management in satellite networks. There, each satellite decides locally which of several short-lived links to send traffic over, using a cheap learned policy instead of a global shortest-path table. The same question arises for a named entity: whose knowledge is "visible" and trustworthy here?
Core Claim
The claim is that span-level source routing reaches the entity-classification accuracy of exhaustive lookup at a fraction of its cost, and beats the obvious heuristic of consulting the knowledge base of whatever language the span appears to be in. The heuristic should fail most on entities that carry no language markers. Such "language-neutral" entities keep their surface form across switches, as Word-level Language Identification in Bi-lingual Code-switched Texts documents. They are exactly where the language identifier has nothing to go on. A secondary and explicitly falsifiable claim is about distribution shift. When the system moves to a new language pair, an online routing policy that adapts from sparse feedback should beat a router trained once offline. If it does not, we drop the online component and keep the routing contribution alone.
Why this, and not the obvious bridge
The research file lists no prior work that joins high-mobility network management with code-switched entity recognition. The tempting bridges are to treat self-attention as a conflict graph and prune it, or to build time-extended graphs of language switches. Both are loose analogies. Reviewers would reasonably answer them with "use an off-the-shelf sparse-attention model" or "just read the whole sentence."
What transfers concretely from the networking work is a decision structure. SkyLink supplies the method. It has each satellite run an upper-confidence-bound bandit over its outgoing links, meaning a learner that tries each option and favors those that have paid off while still exploring. Continuous context is turned into discrete cells by overlapping grids ("tile coding"). SkyLink reports 95% fewer drops than k-shortest paths and constant compute as the constellation grows.
The networking literature also constrains the design. Heavily trained offline policies such as GATE-HPPO and the double deep Q-network scheduler in Service Function Chain Dynamic Scheduling in SAGIN are tuned to one constellation, and the file notes they degrade when reality departs from simulation. This is the offline-versus-online tension our shift experiment tests.
On the language side, three lines of work motivate the proposal:
- MultiCoNER shows plain XLM-RoBERTa, a widely used multilingual encoder, at 54% macro-F1 on complex entities. GEMNET, which merges Wikidata name lists into the encoder through a learned gate, rises to about 84%. Knowledge matters enormously, but GEMNET always performs the lookup and gates only afterwards. - NLPeople at SemEval-2023 retrieves Wikipedia titles, summaries and infoboxes for every span. It finds that finding span boundaries is robust (unlabelled F1 above 90% in 11 of 13 languages), while assigning the entity type degrades under noise. - CMB AI Lab at SemEval-2022 likewise separates boundary detection from classification.
Our proposal leaves the robust first stage alone and targets the fragile second stage and its knowledge supply. The difference from GEMNET and NLPeople is selective, per-span, per-language source choice under a cost budget, rather than uniform always-on retrieval.
Research Program
We build a two-stage recognizer:
- Stage 1 is a category-agnostic span detector, a scorer over pairs of start and end tokens placed on XLM-RoBERTa-large, following CMB AI Lab.
- Stage 2 classifies each span from its encoder representation plus whatever retrieved context the router supplies.
The knowledge sources are separate indexes built from each relevant language's Wikipedia edition, plus a shared Wikidata name list. For English–Spanish these are English and Spanish Wikipedia. For Modern Standard Arabic–Egyptian Arabic they are Arabic and English Wikipedia, the latter reached through transliterated queries. The action set for each span is to consult one source, consult the name list, or skip lookup.
The router sees a small feature vector for the span:
- the span length;
- the stage-1 confidence;
- the stage-2 classifier's entropy (how unsure it is) before any lookup;
- the script of the span;
- the per-token language-ID probabilities inside the span and in a window around it;
- whether the span sits at a switch point.
These features are tile-coded as in SkyLink. The router maximizes the per-span reward
$$r = \mathbb{1}[\hat{y} = y] - \lambda\, c(a),$$
where $\hat{y}$ is the type predicted after taking action $a$, $y$ is the gold type, $c(a)$ is the lookup cost of $a$ (zero for skipping, measured wall-clock latency otherwise), and $\lambda$ trades accuracy against cost. Sweeping $\lambda$ traces an accuracy–cost curve.
We train two routers:
- An offline supervised router. On labelled training data we can compute the reward of every action for every span, so this router is trained directly on those outcomes.
- An online bandit router. It starts from a weak prior and updates from feedback while it is being used.
The online router gets two kinds of feedback. The first is a trickle of gold labels, modelling delayed annotation. The second is a label-free proxy: whether the Wikidata type of the retrieved page agrees with the classifier's prediction. We report the proxy separately because it is partly circular.
Baselines
Naive reference. The do-nothing point is an XLM-RoBERTa token classifier with a label-transition layer (a CRF) and no knowledge. On the MultiCoNER 2022 code-mixed track, CMB AI Lab reports 79.7 F1 for this configuration. On CALCS 2018 there is no identical baseline in the file. The shared-task bests are 63.76% F1 on English–Spanish and 71.61% on Modern Standard Arabic–Egyptian Arabic (CALCS 2018 overview). We report these for context, not as matched comparisons.
Strongest existing system. This is CMB AI Lab's two-stage system, which reports 84.62 F1 on the code-mixed track (third place). We reproduce its single-model version without ensembling and adversarial training, so that every system shares one encoder, one training set and one tuning budget of equal hyperparameter trials. The ensembled 84.62 is quoted as the published ceiling.
Always-retrieve systems. GEMNET is the strongest knowledge-heavy competitor. The file gives about 84% macro-F1 on MultiCoNER overall and no reported result found on Lacuna for GEMNET on the code-mixed track specifically, so we re-run it. NLPeople-style exhaustive retrieval from all sources for every span is the accuracy upper reference at maximum cost.
Language-ID heuristic. This baseline queries the source matching the majority predicted language of the span.
Ablations. Three ablations isolate the key ingredient:
- removing the context language-ID features (does context-sensitivity matter?);
- restricting the router to "skip vs. single shared source" (does per-language choice matter, or only whether to look up?);
- replacing the router with a threshold on classifier entropy.
What beating means. Beating the field requires two results together. First, matching exhaustive retrieval within one F1 point while using at most half its lookups. Second, beating the language-ID heuristic on the language-neutral subset of entities by a margin larger than the spread across random seeds.
Why not just…?
Why not just always retrieve from every source? Exhaustive lookup is the natural upper bound on accuracy, and it may well be the most accurate option. Our claim is about cost. There is also a possible accuracy effect: pages retrieved in the wrong language may act as distractors. The file contains no evidence either way on distractors, so this is an open question the experiment answers. If exhaustive retrieval is both best and cheap enough in practice, only the budget-constrained regime remains as our contribution, and we will say so.
Why not just consult the knowledge base of the span's detected language? This is the most serious threat. For entities with clear language markers, such as "Estados Unidos," it probably works. The prior evidence against it is that code-switched entities are often language-neutral or hybrid forms. The UT System CALCS submission catalogues these, and Jhamtani et al. show they confuse token-level language identifiers. The oracle analysis below measures directly how often the best source differs from the detected language. If that happens rarely, the idea is not worth pursuing.
Why not just threshold on classifier uncertainty, and why not train the router offline? An entropy threshold decides whether to look up but not where, so it is exactly the ablation above. If it matches the full router, per-language routing adds nothing. An offline supervised router is a genuine competitor to the online bandit, since full-information training data exists. The bandit earns its place only if it recovers faster after a shift to an unseen language pair. We test this directly, and we drop it if it does not.
Proposed Experiments
Datasets. We use the MultiCoNER 2022 code-mixed track (task overview) and both CALCS 2018 pairs. For noise stress on stage 2 we use the noisy subsets of MultiCoNER v2.
Metrics. We report entity-level macro-F1, F1 on Creative Work and Title entities (which trail other classes by 10–20 points), F1 on a language-neutral subset, lookups per sentence, and latency.
First decisive check: oracle routing. For every gold span we compute which single source yields a correct type. We then compare three things: the oracle single-source F1, the language-ID heuristic, and exhaustive retrieval. The project continues only if the oracle sits clearly above the heuristic, with the gap concentrated on language-neutral and hybrid entities.
Second check: accuracy–cost curves. We plot F1 against lookups for the router, the entropy threshold, and exhaustive retrieval under matched budgets.
Third check: distribution shift. Routers are trained on MultiCoNER monolingual and multilingual data. They are then deployed as a stream on CALCS English–Spanish and on Modern Standard Arabic–Egyptian Arabic. There, clitics fused to entities (noted in the CALCS overview) change which lookups succeed. We compare the frozen offline router with the online bandit under gold-trickle feedback and under proxy feedback, reporting F1 as a function of the number of feedback events.
References
- Word-level Language Identification in Bi-lingual Code-switched Texts
- SkyLink: Scalable and Resilient Link Management in LEO Satellite Networks
- Graph-Aware Temporal Encoder-Based Service Migration and Resource Allocation in Satellite Networks
- Service Function Chain Dynamic Scheduling in Space-Air-Ground Integrated Networks
- MultiCoNER: A Large-scale Multilingual Dataset for Complex Named Entity Recognition
- NLPeople at SemEval-2023 Task 2: A Staged Approach for Multilingual Named Entity Recognition
- CMB AI Lab at SemEval-2022 Task 11: A Two-Stage Approach for Complex Named Entity Recognition via Span Boundary Detection and Span Classification
- Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task
- The University of Texas System Submission for the Code-Switching Workshop Shared Task 2018
- SemEval-2022 Task 11: Multilingual Complex Named Entity Recognition (MultiCoNER)
- MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition