A patient who asks a drug question in Igbo, Telugu or Yoruba is usually served by one of two pipelines. In the first, the question is machine-translated into English, answered by a strong English model that reads English references, and the answer is translated back. In the second, a multilingual model reads the question directly. Both pipelines have the same weak point: the words that matter most medically are the ones most likely to be damaged when text crosses a language boundary. These are drug names, conditions, doses and contraindications. The pharmacist-evaluated Nigerian drug-information system that uses this cascade reports only 60% drug-information accuracy in Igbo after translation (Retrieval-Augmented Generation Meets Local Languages). On Indic healthcare queries, translating to English preserves the user's coarse intent but leaves clinical entity extraction at 63–68% F1 (Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages).
We propose to treat medical entities as language-independent anchors that are carried unchanged through the whole retrieve–answer–translate loop. Each medical mention in the question is linked to an identifier in a shared medical ontology, meaning a language-neutral concept ID such as a UMLS code. Those IDs are used as an extra retrieval key over both the English corpus and the local corpus. The final vernacular answer is then checked by linking it back to concept IDs and comparing them, along with numeric doses, to the English answer it came from. If the two disagree, the system regenerates the answer or flags it. In two sentences: keep the English-bridge pipeline, because English models and references are much stronger, but stop trusting translation with the entities. Pin every entity to a concept ID, and verify that the same concepts and numbers come out the other side.
Core Claim
Our hypothesis is that in domain-specific cross-lingual retrieval-augmented generation, most of the gap between English and low-resource-language performance comes from entity and number corruption at the language boundaries. Retrieval failure and cross-lingual reasoning account for less. Retrieval-augmented generation (RAG) here means an LLM that answers from passages fetched out of an external document store. If the hypothesis holds, two things should follow. First, a light concept-anchoring layer should recover a large share of the gap, and only on entity-dense questions. Second, a concept-set agreement score between the English answer and the vernacular answer should predict clinical errors well enough to serve as a safety filter.
The claim is falsifiable. If the gains appear equally on entity-light questions, or if errors trace mainly to retrieval, the hypothesis is wrong and the field should invest in retrievers instead.
This places the work against a specific split in the literature. One group of cross-lingual RAG studies is general-domain. XOR QA showed that pivoting to English Wikipedia raises answer coverage by 22% on average. Sentri trained a shared multilingual retriever with a fusion reader. XRAG documented that models answer in the evidence language and struggle to reason over mixed-language evidence. All three use Wikipedia or news, where a garbled entity costs a few F1 points rather than causing a dosing error.
A second group is monolingual medical RAG. MedQA and KMQA show that textbook retrieval is a bottleneck and that structured knowledge helps. They never cross languages.
The method itself comes from CODER. CODER learns cross-lingual medical term embeddings by contrastive training on the UMLS knowledge graph and normalizes terms without translating them. We reuse it as the concept linker. CODER stops at term normalization. We put it inside retrieval and generation and use it as an output verifier.
Research Program
Step 1: link concepts in the question. For each question in language $L$, we extract medical mentions with an LLM prompted for spans. We then link them to UMLS concepts with CODER. For languages CODER was not trained on, such as Indic and Nigerian languages, we use a fallback. The LLM proposes English renderings of each span, and CODER links those renderings. We report linker accuracy separately using the gold entity annotations in the IHQID data, so the reader can see whether linking quality limits the whole approach.
Step 2: retrieve over both corpora. Retrieval runs over a combined index of an English reference corpus and a local-language corpus. We use multilingual-e5 dense retrieval as in the Nigerian system, and add a concept-match score that rewards passages containing the linked concept IDs. Concept tags on passages are computed offline with the same linker.
This step is where cross-lingual information asymmetry enters. Local formulary facts, such as a locally marketed brand of a drug, often exist only in local-language text, while mechanism and contraindication facts exist mainly in English. A concept key can join the two sources even when the surface strings share nothing. BORDIRLINES motivates this design: it found that retrievers default to the query language and that retrieving across relevant languages reduces bias.
Step 3: generate in English, then translate with locked anchors. The English answer marks each anchored concept and each number. Before translation, those spans are replaced with placeholders. After translation, each placeholder is filled with a vetted local surface form taken from a bilingual concept lexicon, built from UMLS multilingual entries and local formularies. If no vetted local form exists, the international name is kept with a short local gloss.
Step 4: round-trip verification. The vernacular answer is linked back to concept IDs. Let $C_{en}$ be the concepts and numbers in the English answer and $C_{L}$ those recovered from the vernacular answer. We compute the agreement
$$F = \frac{2\,|C_{en}\cap C_{L}|}{|C_{en}|+|C_{L}|}.$$
If $F$ falls below a threshold $\tau$, the system retranslates the answer, or abstains and flags it for human review. The threshold $\tau$ is tuned on a development split.
The same score $F$ also serves as an evaluation metric. Accuracy alone does not capture it, and it measures the failure the Nigerian pharmacist study observed.
Out of scope. Speech output is not addressed. Text-to-speech in tonal languages fails for different reasons, such as Yoruba at 40% in the Nigerian study. We note it only as a downstream constraint: anchors at least ensure the text handed to a speech synthesizer is correct.
Baselines
Naive reference. A multilingual LLM answers the native-language question with no retrieval. This shows how much any grounding helps.
Strongest existing pipeline. The English-bridge cascade combines the strategy of Mullick et al. with the pipeline of the Nigerian drug RAG system. It translates the query with NLLB, runs English RAG with multilingual-e5, generates with the same LLM we use, and translates the answer back. Mullick et al. report that English bridging beats direct multilingual modeling on these query types. The only domain-specific figures in the file for this cascade are 73% average text-translation accuracy and 60% Igbo drug-information accuracy (Local Drug RAG), judged by pharmacists on prompts we may not be able to obtain exactly. To beat this baseline, we need a significant gain in pharmacist-judged accuracy and in $F$ on entity-dense questions, with no loss on the rest.
Direct multilingual RAG. Mixed-language retrieval feeds the LLM, which answers directly in the query language. This is XRAG's multilingual setting applied to medicine. It tests whether a capable modern model even needs the round trip. No reported domain result was found on Lacuna for this configuration.
Terminology-constrained translation. The cascade translates with an LLM given a glossary of the linked concepts' local names, with no placeholders and no verification. This is the most dangerous competitor.
Ablations. These isolate each ingredient: concept-keyed retrieval alone, placeholder translation alone, and verification alone.
Fairness controls. Every system uses the same generator LLM, the same corpora and indexes, the same translation model where translation is used, and the same number of LLM calls per question. Where verification adds calls, baselines get equal extra calls through self-consistency sampling. All prompts get the same tuning budget on a shared development split.
For the multiple-choice track, the reported reference points are MedQA's monolingual numbers: 36.7% on USMLE (English) with BioBERT-Large, 42.0% on TWMLE (Traditional Chinese), and 70.1% on MCMLE (Simplified Chinese) with RoBERTa-Large (MedQA). KMQA's 61.8% on the pharmacist exam (KMQA) is the structured-knowledge reference. No reported result was found on Lacuna for cross-lingual retrieval on MedQA.
Why not just…?
Why not just use a better translator, a frontier LLM instead of NLLB? Better translation will close some of the gap, and we run it as a baseline. Translation quality is not the same thing as entity fidelity, though. The Mullick et al. finding that intent survives translation while entities do not suggests that the most fluent rewordings still alter rare medical terms. The experiment that settles this is to compare a frontier-LLM translator against NLLB under the same pipeline and report $F$ separately from fluency.
Why not just give the translator a glossary? This is a genuine threat. It may capture most of the benefit of placeholder translation. The distinct contributions it cannot provide are concept-keyed retrieval, which reaches local-only facts, and round-trip verification, which catches errors after the fact, including corrupted dose numbers that a glossary cannot list. If the glossary baseline matches the full system on accuracy, we will report that. The proposal would then reduce to "glossary translation plus a verifier," and the verifier's value would be judged by how well it catches errors at a fixed abstention rate.
Why not just let a strong multilingual model read mixed evidence and answer natively? This avoids two translation steps. XRAG reports that models drift into the evidence language, failing response-language correctness in over 12% of German cases for Mistral-Large, and that relative accuracy drops can exceed 50% in Arabic (XRAG). Those results come from news, not medicine, and from mid-resource languages, so the direct-multilingual baseline decides the question empirically. The verifier applies to this baseline too, since its output can also be linked back to concepts and checked.
Proposed Experiments
Track 1: cross-lingual MedQA. We build this from existing MedQA resources. TWMLE and MCMLE questions are answered by retrieving from the 18 English textbooks plus the matching Chinese textbooks, and USMLE questions by retrieving from the Chinese textbooks. This creates asymmetric settings with gold answers and no new annotation. The key comparison is cross-lingual-plus-native retrieval against native-only retrieval, under each baseline.
Track 2: open-ended consumer health questions. These come from IHQID-WebMD, IHQID-1mg and the real-world hospital queries in seven Indic languages (Mullick et al.). Their gold entity annotations let us compute linker accuracy and an entity-recall version of $F$. Licensed pharmacists will rate a stratified sample for correctness and harm, blind to which system produced each answer.
Track 3: Yoruba, Igbo and Hausa drug questions. We will request the Nigerian pharmacist prompts. If they are unavailable, we will recreate a comparable set with pharmacists, using a combined English and Nigerian corpus.
Control: general-domain QA. XOR-TyDi QA (best reported XOR-Full F1 of 18.7, XOR QA) serves as a control where entities are not mostly medical. Here we expect small or no gains from anchoring.
Decisive check 1: where do errors occur? We attribute every error to one stage: linking, retrieval, English reasoning, or boundary corruption. The attribution compares each vernacular output with its own English intermediate answer and with gold. The core claim survives only if boundary corruption is the largest share of the gap between English and the other languages for the cascade baseline, and anchoring removes most of it.
Decisive check 2: are gains specific to entities? We split questions by number of linked concepts. Gains should grow with entity density and vanish on intent-only questions and on XOR-TyDi.
Decisive check 3: is the verifier useful for safety? At a fixed abstention rate of 10%, the verifier should catch a clearly larger fraction of pharmacist-flagged harmful answers than the generator's own confidence or random abstention.
Decisive check 4: does linking hold up in low-resource languages? If linker accuracy on IHQID entities falls far below the cascade's own entity F1, the anchors themselves become the bottleneck. In that case the honest conclusion is that cross-lingual medical linking for these languages, not RAG design, is the real open problem.
References
- Retrieval-Augmented Generation Meets Local Languages for Improved Drug Information Access and Comprehension.
- Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages
- XOR QA: Cross-lingual Open-Retrieval Question Answering
- Ask Me Anything in Your Native Language
- XRAG: Cross-lingual Retrieval-Augmented Generation
- What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams
- Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text
- CODER: Knowledge infused cross-lingual medical term embedding for term normalization
- Multilingual Retrieval Augmented Generation for Culturally-Sensitive Tasks: A Benchmark for Cross-lingual Robustness