Ontology-Checked Alias Growth for Non-English Clinical Entity Recognition

Hospitals outside the English-speaking world have plenty of clinical notes, and almost none of them are labeled. A cheap way to label them is distant supervision: take a dictionary of medical terms, mark every place a term appears in the notes, and train a tagger on those automatic labels. The approach has worked in English, where the Unified Medical Language System (UMLS, a large merged vocabulary of medical concepts and their names) is rich. For other languages the dictionary is the weak link. Concept coverage is not the main problem, because Wikidata now links German names to 98.2% of the concepts needed. The problem is the words clinicians actually write: those Wikidata-derived names cover only 56–63% of the surface forms found in real biomedical text (Leveraging Wikidata for Biomedical Entity Linking in a Low-Resource Setting). A tagger trained on dictionary matches therefore learns that abbreviations, brand names, and telegraphic shorthand are not entities.

We propose to grow the target-language dictionary from the hospital's own unlabeled notes. The tagger nominates spans the dictionary missed. A span is admitted as a new alias only if three checks pass: it links confidently to a known concept, that concept's semantic type agrees with the tagger's label, and an English rendering of the span links to the same concept in the much richer English UMLS. The admitted aliases become new dictionary entries for the next round of labeling. The ontology plays the role that an anatomical shape model plays in semi-supervised landmark detection: it is a fixed outside structure that rejects implausible pseudo-labels before they can reinforce themselves. A budget of roughly fifty expert-annotated sentences per language is spent only on calibrating the noise model and the acceptance threshold, and none of it goes into training the tagger.

Core Claim

In non-English clinical named entity recognition (NER, finding and typing mentions such as drugs and disorders), the main loss from distant supervision comes from missing clinical aliases, more than from noisy matches. Adding aliases from unlabeled in-domain notes, under an ontology-consistency check, closes more of the gap to fully supervised training than three cheaper options: better noise modeling over static dictionaries, transfer from English gold data, or spending the same expert budget on direct training. The claim is falsifiable at two levels. First, alias coverage of held-out gold mentions should rise measurably above the Wikidata baseline. Second, the resulting tagger should beat the strongest transfer baseline when that baseline is folded into the same pipeline rather than run in isolation.

How This Differs From the Closest Work

The nearest work is Trove (Ontology-driven weak supervision for clinical entity classification in electronic health records). Trove turns UMLS into labeling functions, which are small programs that each vote on a label. It combines their votes with an unsupervised label model, a statistical model that estimates each voter's accuracy without seeing gold labels, and trains BioBERT on the result. It came within 1.3–4.9 F1 of supervised models in English. Trove supplies our noise-modeling method. Its dictionaries are static and English, though, and it never asks what happens when the dictionary itself is the bottleneck.

ANEA (ANEA: Distant Supervision for Low-Resource Named Entity Recognition) builds Wikidata gazetteers for low-resource languages and has users tune matching rules by hand. It shows that the gazetteer route works across languages, but it targets general-domain entities and fixes noise with manual heuristics rather than ontology constraints.

The German Wikidata study and BabelNet synonym expansion (Exploiting BabelNet for Multilingual Biomedical Synonym Expansion) supply our starting dictionaries. They also motivate the proposal, since both mine encyclopedic sources and neither touches clinical text, which is exactly where the alias deficit appears.

Con2GEN (Controllable Contrastive Generation for Multilingual Biomedical Entity Linking) creates multilingual training data by swapping foreign synonyms into English sentences. That keeps English syntax intact. We instead mine aliases from authentic target-language notes.

The idea of rejecting pseudo-labels with a structural prior comes from Semi-supervised Anatomical Landmark Detection via Shape-regulated Self-training. That work projected landmark predictions onto a PCA shape model, rejected outliers, and beat FixMatch by 3.8–6.3%. We translate that idea from shape space into a concept graph. Finally, SNOKind (Deterministic-First Clinical Concept Normalization in Brazilian Portuguese) shows that placing deterministic matching first is cheap and auditable. We keep that ordering. SNOKind, however, never feeds its residual cases back into the dictionary, and it reports a resolution rate rather than accuracy against gold.

Research Program

The pipeline starts with a seed dictionary per language. For German we use UMLS_Wikidata. For French, Spanish, and Dutch we build the equivalent by querying Wikidata through the UMLS concept-identifier property P2892, as the German study did. Where available we union in BabelNet expansions and the native UMLS subset; the Spanish subset is roughly seven times smaller than English (Biomedical term normalization of EHRs with UMLS).

Labeling functions include longest-match dictionary lookup, abbreviation rules, a semantic-type filter, and, as optional extra voters, an English-transfer tagger and a locally hosted open LLM. Their votes go into a Trove-style label model. We then fine-tune XLM-R Large on the label model's probabilistic labels over the unlabeled notes.

Each growth round has the tagger propose spans that it labels with confidence but that the dictionary does not contain. Each proposed span $s$ is embedded with multilingual SapBERT and retrieved against the union of target-language and English concept names. Let $c^\*$ be the top concept and $m(s)$ the cosine margin between the first and second candidates. The span is admitted only if all three of the following hold:

$$ m(s) \ge \tau, \quad \mathrm{type}(c^*) \in T(y), \quad \mathrm{link}_{EN}(\mathrm{tr}(s)) = c^* $$

In this condition, $\tau$ is a margin threshold, $y$ is the tagger's predicted label, $T(y)$ is the set of UMLS semantic types allowed for that label, $\mathrm{tr}(s)$ is a machine translation of the span plus a short context window, and $\mathrm{link}_{EN}$ is linking against full English UMLS. The third test is the cross-lingual ingredient. It uses the English side's richer alias inventory to confirm the target-language guess.

Admitted aliases are added to the dictionary, the labeling functions are re-run, and the cycle repeats until fewer than a fixed fraction of new aliases are admitted. The fifty expert sentences tune $\tau$ and the label model's class balance, and nothing else.

Baselines

The naive reference point is dictionary longest-match with the seed dictionary and no learning. It shows what the raw dictionary gives and anchors the alias-coverage measurement.

The second baseline is static weak supervision: the full Trove-style label model and the same XLM-R Large tagger, trained on seed dictionaries with no growth. This is the ablation that isolates our key ingredient. Same data, same tagger, same compute, and the only difference is whether the dictionary grows.

The strongest external competitor is zero-shot cross-lingual transfer: XLM-R Large fine-tuned on English n2c2 2018 Track 2 and applied directly to French MedNERF and German GERNERMED. Multilingual Clinical NER: Translation or Cross-lingual Transfer? found it the best of the zero-target-annotation methods, and found translate-train close behind. We run both with the same encoder and tuning budget. We found no reported F1 on Lacuna for MedNERF or GERNERMED with this method, so we will report our own reproduction.

The fourth baseline is few-shot supervision: XLM-R Large fine-tuned directly on the same fifty expert sentences. It tests whether the expert budget is better spent on training than on calibration.

The ceiling is fully supervised training where gold training data exists. On PharmaCoNER, A Deep Learning-Based System for PharmaCoNER reports NER F1 of 0.9105 with 500 annotated training cases. On the Dutch ADE corpus, inter-annotator strict entity F1 of 0.7724 (Dutch ADE corpus) bounds what any system can meaningfully reach.

Beating the field means three things. Alias growth must beat static weak supervision by a margin larger than the seed-to-seed variance. It must beat zero-shot transfer and translate-train on MedNERF. On PharmaCoNER it must land within the 1.3–4.9 F1 band Trove achieved in English, without using any of the 500 gold training cases.

Two further ablations isolate the check itself. The first is growth without any check, which is plain self-training on unmatched spans. The second is growth with the margin and type tests but no English back-link.

Why not just…?

Why not just use zero-shot cross-lingual transfer from n2c2? This is the real threat. The transfer study found it the strongest zero-target-annotation method for French and German. Its weakness is coverage: n2c2 covers medications and adverse events, so it cannot help with entity types outside that inventory. It also inherits English surface forms, not local brand names or local shorthand. Rather than argue the point, we include the transfer tagger as one more labeling function. If alias growth adds nothing on top of it on MedNERF, the core claim fails for medication-type entities, and we will report that.

Why not just prompt an LLM to annotate the notes? SNOKind shows LLMs can do clinical extraction in Portuguese. It also shows that a standalone LLM resolved only 57% of entities, against 92% for its cascade, and that the cascade's quality was never checked against gold. Clinical notes also generally cannot be sent to external services. We therefore run a local open LLM both as a labeling function and as a standalone annotator under the same compute budget. Our expectation is that it adds recall on abbreviations but needs the ontology check to keep precision, which the ablations will show directly.

Why not just annotate a few hundred target sentences? If a modest gold set beats the whole pipeline, that is the honest answer for practitioners. The Dutch ADE effort shows why it is rarely cheap: double annotation by trainees still needed senior physician and pharmacist adjudication. We test this directly. We plot few-shot supervised F1 on PharmaCoNER at 50, 100, 250, and 500 cases and find the number of gold cases at which it overtakes alias growth. That crossover point is a headline result whichever way it falls.

Why not just use better noise modeling over the existing dictionaries? The static weak-supervision ablation answers this. If the label model alone recovers most of the gain, then noise, not missing aliases, is the bottleneck and our diagnosis is wrong.

Proposed Experiments

The main testbed is Spanish PharmaCoNER. It has 3,751 unannotated background cases to serve as the growth pool, a 250-case gold test set, and a supervised ceiling, so every comparison is clean. Dutch ADE (102 ICU notes, drug and disorder entities) tests the same pipeline on ICU text. We run it with cross-validation in which test notes are never part of the growth pool, and we state plainly that this pool is small. French MedNERF and German GERNERMED are zero-gold transfer checks. GERNERMED's 30 sentences make it a sanity check, not a basis for claims. For linking quality of the admitted aliases, we use the WikiMed-DE-BEL setup, where Jina embeddings reach P@1 of 0.805 over UMLS_Wikidata.

We report strict and lenient entity F1, plus two quantities specific to the mechanism. The first is alias coverage: the fraction of held-out gold mention strings present in the dictionary, tracked before and after each growth round against the 56–63% Wikidata starting point. The second is alias precision: a clinician audits a random sample of 200 admitted aliases per language.

Three outcomes would be decisive. Coverage should rise and F1 should rise with it, while the unchecked self-training ablation's alias precision collapses across rounds; that is the confirmation-bias signature the ontology check is meant to stop. Removing the English back-link should measurably lower alias precision. And alias growth, with the transfer tagger included as a voter, should beat the transfer tagger alone on MedNERF.

If coverage rises but F1 does not, the missing-alias diagnosis is wrong and the paper becomes a negative result about where non-English distant supervision actually loses.