Do Domain-Tuned Translators Undo Corrections?

When a journal issues a correction, the change is usually tiny: an author's name respelled, an affiliation swapped, a percentage range changed from "40–115%" to "30–55%", two figure labels exchanged. Work on the scholarly record treats these edits as harmless to the science, because findings survive them. That is true for the paper. It is not obviously true for the machine translation systems that carry the paper into other languages. A translator fine-tuned on a narrow scientific domain learns strong expectations about which names, numbers, and term combinations usually appear. When the corrected text contains something rarer than the original, the translator may quietly write the old or more typical version back in. This proposal tests whether in-domain fine-tuning makes translators worse at carrying small corrective edits into their output. It uses real correction notices as a natural source of before-and-after sentence pairs, each labelled by what kind of edit it is.

In brief: correction notices hand us thousands of minimal pairs, meaning two versions of a sentence that differ by exactly one meaningful edit. We translate both versions and check whether the difference between the two translations matches the edit. Standard translation scores like BLEU (word-overlap with a reference translation) barely move when one number or name is wrong. So the main risk of domain specialization, silently reverting or smoothing corrections, is currently invisible to the evaluations that justify fine-tuning.

Core Claim

We claim that in-domain fine-tuning raises what we call reversion. Given an original sentence $s$ and its corrected version $s'$, with translations $t = M(s)$ and $t' = M(s')$ by model $M$, reversion happens when $t'$ contains the pre-correction value or a more frequent in-domain alternative instead of the corrected one. We define edit fidelity as the fraction of pairs where the change from $t$ to $t'$ realizes the source edit and nothing else. The prediction is that fidelity falls as fine-tuning becomes more aggressive (larger learning rate, more epochs, smaller and more repetitive in-domain sets). It should fall most for edit types the correction literature files as "administrative": name spellings, author roles, affiliations. These spans carry little domain signal, so the model's domain prior can overwrite them.

The claim is conditional. If fidelity does not degrade, that is a useful negative result: domain adaptation's known cost on general text does not extend to fine-grained edit transmission. The project also delivers a reusable diagnostic either way.

Why This Intersection, and What Is Already Known

The correction literature supplies both the motivation and the data. It documents a consistent split between administrative edits and localized substantive edits. On the administrative side are author-name corrections in Belkaid et al., Le Franc et al. and Yang et al., a restructured author list in Ahn, affiliation fixes in Lupperger et al., and funding disclosures in Zhang et al.. On the substantive side are numeric range corrections in Song et al., swapped haplotype axis labels in Bevers et al., and an AUC corrected from 0.97 to 0.91 in Muchlinski et al.. These papers report the edits but never ask what happens to them downstream. Cheng et al. found that over 30% of retraction notices give no clear rationale, and Shen documents notices that never reach digital readers. Both findings constrain us. We cannot rely on notices to state their edit type cleanly, so we label edit types ourselves from the diff.

The fine-tuning literature motivates the mechanism.

  • Small in-domain sets move models a lot. McNamee and Duh get most of their gain from about 1,000 examples. Akinade et al. lift M2M-100 from 4.33 to 29.67 BLEU with under 1,000 pairs.
  • The movement has side effects. General-domain BLEU fell from 41.0 to 34.74 in Optum's WMT22 submission, and Flores-101 dropped 5.8% for McNamee and Duh.
  • Priors can overpower the source text. The "death bias" Akinade et al. document shows a frequency prior overriding the intended reading of a word. We hypothesize that fine-tuning installs a new, domain-specific prior of the same kind.
  • Multi-word institutional terms are already fragile. MAGMATic shows accuracy near 53.65% on multi-word terms versus over 76% on single words, with institutional and administrative terms harder than disciplinary ones.

None of these works measures sensitivity to minimal source edits. Their metrics could not detect it, and Zouhar et al. show that learned quality metrics are unreliable in unseen specialized domains. That rules out a learned metric as our judge.

The closest bridging work in the file does not touch this question. MAGMATic evaluates administrative terminology as static text. Optum uses administrative insurance letters purely as domain data. DomainInv "corrects" noisy pseudo-labels during adaptation, which is unrelated to scholarly corrections. MTUncertainty predicts post-editing effort, not edit transmission.

We rejected one tempting design and should say so. It would filter uncorrected "window-of-risk" versions out of fine-tuning corpora. That idea fails for translation. A parallel pair in which both sides carry the same wrong number is still a correct translation, so it is not label noise. Pairs with mismatched versions are caught by a trivial check that numbers match across source and target. The real risk sits at inference time, not in training data.

Research Program

Benchmark construction. We will build a correction minimal-pair benchmark. From publisher correction notices, starting from the kinds indexed above and extending to a few hundred English-language notices, we extract original and corrected sentence spans and label each diff with one edit type:

  • name spelling
  • author order or role
  • affiliation
  • funding or ethics statement
  • numeric value or range
  • label swap
  • multi-word term change

Real notices give authenticity but modest volume. We therefore add templated pairs. We apply the same edit types, with their empirical distribution from the real notices, to held-out sentences of the WMT19, WMT22, and WMT23 biomedical test sets and of MAGMATic. Each templated edit is validated by a bilingual annotator. Keeping real and templated pairs as separate subsets lets us check that conclusions do not rest on synthetic edits.

Scoring. Each pair's two translations are scored by automatic diff alignment plus human adjudication of ambiguous cases. There are four outcomes:

  • faithful: the target diff realizes the edit
  • non-propagation: $t' = t$ on the edited span
  • reversion: $t'$ restores the original, or a value or spelling more frequent in the fine-tuning data
  • spillover: other spans change

To test the prior-pull mechanism, we regress reversion on how often the original and corrected strings appear in the fine-tuning corpus.

Dose-response sweep. We fine-tune opus-mt-en-es on biomedical data following Optum's recipe. We vary learning rate (their $1e-6$ versus mixed-domain $1e-5$), set size (1k, 10k, full), and epochs. We repeat the sweep with M2M-100 (418M) for EN→PT and EN→DE, using fairseq.

Remedy. The candidate fix is correction-typed contrastive augmentation. During fine-tuning we add paired copies of in-domain training sentences with the same edit applied consistently to source and target, drawn from the empirical edit-type distribution. We train with an extra loss that rewards the translation difference for tracking the source difference.

Baselines

Unadapted general model (the do-nothing reference). This is the general model with no in-domain training: opus-mt-en-es, which Optum reports at 36.89 BLEU on the WMT22 test set, and base M2M-100. It fixes the fidelity level that fine-tuning is supposed to preserve. It runs on identical minimal pairs with identical decoding (beam size 5, no length penalty tuning).

Optum-style in-domain fine-tuning (the strongest reported adaptation). Conservative fine-tuning at $1e-6$ on clinical and biomedical data reached 38.12 BLEU on WMT22 and 58.86 on EMEA/M&R letters in Optum's submission. We also run a data-selection variant from HW-TSC: LaBSE filtering at 0.7 plus domain-score curriculum, which they credit with +3.0 SacreBLEU. We do not reproduce their 25-layer model. All variants get the same training data pool, the same GPU-hours, and the same number of hyperparameter trials selected on a held-out development split of minimal pairs. There is no reported edit-fidelity number for any of these systems on Lacuna, so we establish their fidelity ourselves.

Placeholder masking. Numbers and named entities are replaced by tags before translation and restored afterwards. This is the cheapest engineering answer to numeric and name reversion, and it runs on the same fine-tuned checkpoints.

Lexical constraints. Following the HOLLY work and its codebase, corrected terms are forced into the output with semantically appropriate constraint application.

Ablation of the remedy. We run the same augmentation volume with uniformly random perturbations that ignore the correction taxonomy, and a second version that drops the contrastive loss. Together these isolate whether realistic edit types and the paired objective matter, rather than just seeing more perturbed data.

What counts as winning. The remedy beats the field if it matches Optum-style fine-tuning on in-domain BLEU (within 0.5) while restoring edit fidelity to at least the unadapted model's level. It must also beat placeholder masking on the edit types masking cannot reach: affiliations, multi-word term changes, and label swaps.

Why not just…?

Why not just mask numbers and names with placeholders? For pure numeric corrections like Song et al.'s ranges, masking may well solve reversion, and we expect it to be a real threat there. It does not cover affiliation strings, multi-word institutional terms, or label swaps that are ordinary words. MAGMATic's multi-word result suggests these are exactly where domain priors bite. It also fails when entity recognition misses a transliterated or misspelled name, and name misspellings are the most common correction in the notices above. The per-edit-type breakdown settles this. If masking closes the gap on every type, we report that the diagnostic stands and the remedy is unnecessary.

Why not just run a source–target consistency check at output time? A digit-matching check flags numeric reversions and can trigger re-decoding. It cannot tell a correct translation of a respelled name from a "normalized" one, and it has nothing to compare for word-level label swaps. We include it as a post-hoc filter on all systems and report the fidelity it recovers per edit type.

Why not just keep the general model for correction notices and use the adapted one elsewhere? This routing is sensible if the general model is more faithful. But corrections are embedded in ordinary domain sentences, so routing would give up the in-domain gains Optum and MAGMATic report on exactly the text being corrected. Our dose-response curves quantify the trade: how much BLEU is lost by routing versus how much fidelity is lost by adapting.

Why not just rely on BLEU or a learned metric to catch this? A single wrong number moves sentence BLEU by a few points at most, and Zouhar et al. show learned metrics degrade in biomedical text. We report the correlation between BLEU or learned-metric deltas and adjudicated reversions to show whether they detect the failure at all.

Decisive Checks

Mechanism. Reversion should rise monotonically with fine-tuning strength and be concentrated on administrative edit types and on strings frequent in the fine-tuning data. If reversion is flat across the sweep, the core claim fails.

Transfer from synthetic to real. Fidelity rankings of systems on templated pairs should match rankings on real correction-notice pairs (Spearman correlation reported). If they disagree, only the real subset is trusted.

Remedy. Correction-typed augmentation should beat the random-perturbation ablation on real pairs at equal in-domain BLEU. If it does not, the taxonomy from the correction literature adds nothing beyond generic robustness training, and we say so.

References

  1. Author Correction: Mice adaptively generate choice variability in a deterministic task
  2. Correction to: EMSx: a numerical benchmark for energy management systems
  3. Correction to: Comparative Genomics Analysis Provides Insights into Pregnancy Maintenance Mechanisms in the Ovoviviparous Teleost Black Rockfish (Sebastes schlegelii)
  4. Correction to: Effect of Aging on Pitting Corrosion Resistance of 21Cr Lean Duplex Stainless Steel with Different Molybdenum Contents
  5. Correction: Reoccuring neural stem cell divisions in the adult zebrafish telencephalon are sufficient for the emergence of aggregated spatio-temporal patterns
  6. Corrigendum: Detection methods of nanoparticles synthesized by gas-phase method: a review
  7. Author Correction: China’s bulk material loops can be closed but deep decarbonization requires demand reduction
  8. Author Correction: Mitochondrial haplotypes affect metabolic phenotypes in the Drosophila Genetic Reference Panel
  9. Seeing the Forest through the Trees
  10. Retractions relevant to COVID-19: does the retraction rate jump during the pandemic?
  11. Correction to: Copyright Volume 31, Issues 1–4, Acta Mechanica Solida Sinica
  12. The Multilingual Microblog Translation Corpus: Improving and Evaluating Translation of User-Generated Text
  13. ε kú : Integrating YorùBá Cultural greetings into Machine Translation
  14. Optum's Submission to WMT22 Biomedical Translation Tasks
  15. MAGMATic: A Multi-domain Academic Gold Standard with Manual Annotation of Terminology for Machine Translation Evaluation
  16. Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains
  17. DomainInv: Domain Invariant Fine Tuning and Adversarial Label Correction For Unsupervised QA Domain Adaptation
  18. MTUncertainty: Assessing the Need for Post-editing of Machine Translation Outputs by Fine-tuning OpenAI LLMs
  19. The Path to Continuous Domain Adaptation Improvements by HW-TSC for the WMT23 Biomedical Translation Shared Task
  20. Towards Accurate Translation via Semantically Appropriate Application of Lexical Constraints