Medical text simplification rewrites dense clinical text, such as trial abstracts, consent forms, or reference-manual entries, into plain language a patient can follow. It is always a trade-off. Each step toward clarity risks losing a clinically meaningful detail, as when "non-small cell lung cancer" becomes "lung cancer." Most of the world's patients do not read English, and most simplification training data is English. The obvious remedy is zero-shot cross-lingual transfer: train a multilingual model on English simplification only, then run it directly in Spanish, Japanese, or Farsi. We argue that this transfer is lopsided. The pressure to simplify (rephrase, shorten, drop jargon) carries across languages, because it is largely structural. The pressure to stay faithful does not, because the training signals that enforce it are tied to English words. The result is a model that becomes looser, not safer, outside English.
We propose to measure this asymmetry directly as a per-language trade-off curve between clarity and fidelity. We also propose a fix. The fix moves the fidelity signal off English tokens and onto a language-independent notion of specificity: whether each medical concept in the source survives at the same level of detail. We learn a specificity checker from English data only, inside a shared multilingual encoder. We use it twice: as a training penalty that lets the model drop jargon forms while keeping concept detail, and as a reranking utility at decoding time in target languages where no labels exist. The claim is falsifiable. If our method is right, the target-language clarity–fidelity curve should sit close to the English curve. With token-level jargon penalties, it should sag.
Core Claim and Its Grounding
Two observations in the literature motivate the asymmetry hypothesis, and they appear to contradict each other. Paragraph-level Simplification of Medical Texts penalizes the probability of tokens flagged as jargon by an English classifier (92.7% accurate). Using this unlikelihood training, it cuts 4-gram copying from 0.79 to 0.49 and Flesch-Kincaid grade from 13.44 to 11.97, below the human summaries at 13.11. In English, more abstraction meant better outputs. Multilingual Simplification of Medical Texts finds the opposite pattern outside English. Zero-shot GPT-3 stays extractive, behaving like literal translation, and stays faithful. Fine-tuned mT5 is simpler but hallucinates and mistranslates acronyms.
Our reading reconciles the two. Abstraction is safe only when a fidelity signal accompanies it. In English that signal is implicit in the token-level penalty's target vocabulary. In other languages it is absent. InfoLossQA shows that 74.1% of human-identified information loss in English simplification is oversimplification rather than deletion. That is exactly the failure a token-level jargon penalty encourages and a specificity-level signal could catch.
The closest prior work does not test this. MultiMSD adds an "active paraphrasing" loss that rewards predicting reference words absent from the source. It trains with target-language parallel data, so it never asks whether a fidelity- or jargon-shaping objective survives zero-shot transfer. Zero-Shot Crosslingual Sentence Simplification (ZEST) does transfer simplification zero-shot, using adversarially language-neutral encoders and English–German translation data. It works on general-domain German and has no fidelity mechanism. Our proposal differs from both. The training signal is about preserved concept specificity, not about tokens. It is learned in English only. It is checked for transfer as a separate, measurable component.
Research Program
The first part is diagnostic. We fine-tune two multilingual generators, mT5 and Llama 3 (the latter matching MultiMSD), on English data only. The English data is the Cochrane paragraph corpus (4,459 pairs) plus the English portion of MultiMSD. We use three objectives: plain cross-entropy, the token-level jargon unlikelihood penalty of Devaraj et al., and MultiMSD's active paraphrasing loss applied to English only. For each objective we sweep penalty strength to trace a curve of clarity against fidelity. We then evaluate the same checkpoints zero-shot on the eight other MultiMSD languages and on the Spanish, French, and Farsi portions of MULTICOCHRANE. The diagnostic quantity is how far each target-language curve sits below the English curve. The hypothesis predicts that the gap widens as jargon-penalty strength increases.
The second part builds the specificity checker. It is a scoring head on a frozen multilingual sentence encoder of the kind MultiMSD used for alignment (LaBSE or mE5). It takes a source concept span and the output and predicts whether the concept was preserved, generalized, or dropped. We train it on English data only. Positives come from InfoLossQA evidence and rationale spans, and from aligned Cochrane spans. Negatives come from controlled generalizations, where we replace a specific term with a hypernym from the same sentence's plain-language counterpart. The design bets that "same concept at the same specificity" is represented more language-neutrally than surface jargon, in line with the shared-geometry evidence behind cross-lingual parsing and QA (75 Languages, 1 Model; Zero-shot Reading Comprehension). The bet could fail. MLQA reports roughly a 40% Exact Match drop when extractive QA transfers out of English, with Hindi at 48.8 F1. A span-matching checker could therefore inherit a large transfer gap, so we measure the checker's own transfer before using it.
The third part uses the checker. During English training we pair the jargon penalty with a specificity-preservation penalty, so the model is pushed off jargon forms but toward keeping concept detail. At inference in a target language we generate diverse candidates and select among them with Minimum Bayes Risk decoding, which picks the candidate that agrees most with the others under a chosen utility. This follows UZH Pandas at SimpleText2024, but our utility combines similarity with the checker's preservation score. That paper used MBR to avoid oversimplification in English. We supply a fidelity utility meant to work where no target labels exist.
Evaluation Data and Metrics
We work at two granularities. At the sentence level we use MultiMSD's nine languages. At the paragraph level we use MULTICOCHRANE's MC-CLEAN, 101 manually aligned abstracts in four languages, which also addresses that paper's stated limitation that sentence-level simplification misses context.
Clarity is measured by SARI, a standard simplification metric that rewards appropriate additions, deletions, and keeps against references. It is supplemented by target-language jargon counts in the style of MultiMSD's English term count (513 source terms, 490 output, 485 reference), with term lists produced by native annotators. We do not rely on readability formulas, which are English-specific.
Fidelity is the outcome that matters. Prior work found that automatic metrics correlate weakly with clinical judgments, and the file has no automatic fidelity metric with known target-language validity. We therefore build a small cross-lingual InfoLossQA extension. Native speakers with clinical training write and verify question–answer pairs for a subset of MC-CLEAN abstracts in Spanish, Farsi, and Japanese, and label outputs for oversimplification, deletion, and hallucination, acronym errors included. This set is also the held-out test of the checker's transfer.
Baselines
The naive reference point is the same multilingual generator fine-tuned on English with plain cross-entropy and applied zero-shot. It shows what transfer yields with no fidelity mechanism at all. A second naive point is untuned zero-shot prompting of Llama 3, for which MultiMSD reports a SARI of 45.21 on Japanese. We reproduce this prompt setting on our splits so all systems share the same test sentences.
The strongest existing zero-shot method on the clarity side is English-only token-level jargon unlikelihood (Devaraj et al.). It is the published mechanism for escaping verbatim copying, and our hypothesis is specifically about its failure to transfer. It gets the same base model, English data, number of training steps, and penalty-strength sweep as our method. ZEST is the strongest existing zero-shot cross-lingual simplifier. It reports 41.12 SARI on TextComplexityDE against 38.64 for a translation pivot, but that is general-domain German. We re-train it on our English medical data plus the same translation data available to the translate-train baseline below.
As a reference ceiling rather than a competitor, we use MultiMSD's supervised multilingual model with active paraphrasing, reported at 50.02 SARI on Japanese. It uses target-language training pairs that zero-shot methods do not see.
Beating the zero-shot baselines means the following. At matched jargon count or SARI, our method has a significantly lower human-annotated oversimplification and hallucination rate in every target language. Equivalently, its target-language curve sits closer to its English curve than the token-penalty curve does. Matching the supervised ceiling on SARI is not required. Matching or beating it on fidelity would be a strong result.
Two ablations isolate the key ingredient. The first uses the specificity penalty in training without checker-based MBR, and the second uses MBR with the checker but without the training penalty. These separate learned behavior from decode-time filtering. A further control replaces the specificity checker with a plain similarity utility of equal compute in MBR, to show that the gain comes from the specificity signal rather than from reranking itself. All MBR variants use the same number of candidates.
Why not just…?
Why not just simplify in English and machine-translate the result? This pipeline is the most practical competitor. Its weakness is documented: Joseph et al. report that translation introduces terminology and acronym errors on clinical concepts, and ZEST beat a translation pivot by about 2.5 SARI in German. Those results are partly general-domain and SARI-based, however, so they do not settle fidelity. We run simplify-then-translate with the strongest English simplifier and score it with the same human fidelity protocol. If it matches our fidelity at equal clarity, the honest conclusion is that cross-lingual fidelity should be handled by translation, and we will report that.
Why not just prompt a frontier LLM, or use one as the fidelity judge? As a generator, zero-shot prompting stayed extractive or became unfaithful in non-English medical text (Joseph et al.). Even GPT-4-Turbo reached only 52.45% adjacent-level accuracy on English readability control (Analysing Zero-Shot Readability-Controlled Sentence Simplification). As a judge, GPT-4 recovered only 28.1% of expert-identified information loss in English (InfoLossQA), so an LLM utility in MBR is a weak fidelity signal before transfer even enters. We nonetheless include an LLM-judge MBR utility as an alternative to our checker. Comparing the two on recall against the cross-lingual annotations decides which signal to use.
Why not just translate the English training data into each target language and train there (translate-train)? This is a real threat, because it gives the model target-language inputs and keeps the token-level jargon penalty applicable. The objections are that translated "complex" sources may already be partly simplified or garbled at exactly the jargon spans, and that it needs a translation system and jargon classifier per language. We test it head-on with the same token-penalty training on translated Cochrane and MultiMSD English data. If translate-train closes the fidelity gap as well as specificity transfer, our contribution narrows to the diagnosis, and the method is unnecessary.
Decisive Checks
Four results settle the proposal. The first is the diagnostic itself. If token-level jargon penalties do not widen the English–target fidelity gap as their strength grows, the asymmetry hypothesis is wrong and the remaining work loses its motivation. The second is the checker's transfer. Its preserved-versus-generalized accuracy on native-annotated Spanish, Farsi, and Japanese pairs must stay well above chance and close to its English accuracy. If it collapses the way extractive QA did in MLQA, the mechanism fails at its foundation. The third is the main comparison: at matched clarity, the oversimplification and hallucination rate must fall relative to the token-penalty and translate-train baselines. The fourth is the paragraph-level result on MC-CLEAN, where context matters most for acronym and finding interpretation. There we expect the advantage over sentence-level pipelines to be largest. Farsi, which is in MULTICOCHRANE but not MultiMSD, is the cleanest test of transfer to a language with no in-domain training data.
References
- Paragraph-level Simplification of Medical Texts
- Multilingual Simplification of Medical Texts
- InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification
- MultiMSD: A Corpus for Multilingual Medical Text Simplification from Online Medical References
- Zero-Shot Crosslingual Sentence Simplification
- 75 Languages, 1 Model: Parsing Universal Dependencies Universally
- Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model
- MLQA: Evaluating Cross-lingual Extractive Question Answering
- UZH Pandas at SimpleText2024: Multi-Prompt Minimum Bayes Risk with Diverse Prompts
- Analysing Zero-Shot Readability-Controlled Sentence Simplification