Borrowing Fact-Checks Across Languages to Repair Fairness Drift in Misinformation Detection

A misinformation detector deployed on a multilingual news feed does not fail evenly. It is usually trained mostly on English, it is fed new rumors whose wording and topics change week to week, and the only feedback it gets about its own mistakes comes from human fact-checks. Those fact-checks arrive quickly and in volume for English and a few large languages, and slowly or never for most others. So the languages where the detector is most likely to drift, wrongly flagging legitimate local news as fake or missing local hoaxes, are also the languages where nobody can see it happening. This note proposes to close that loop. Many false claims are debunked first in one language and then resurface, translated, in others; one cited paper calls these "zombie" claims. When a fact-checked English claim is found again in Urdu or Malayalam, its verdict can stand in as a label in the second language. Those borrowed labels can then drive an online monitor that tracks error rates per language and adjusts decision thresholds per language as the stream moves.

In two sentences: language disparity in deployed misinformation detectors is a fairness-drift problem, and the fix is starved by uneven fact-check feedback. We propose to transfer verdicts across languages by matching claims, and to use those transferred verdicts, pooled with each language's own scarce labels, to detect and repair per-language error-rate drift online without retraining the detector.

Core Claim

The claim has two testable parts. The first is that the gap between high-resource and low-resource languages is not a fixed number measured once on a test split. It reopens and widens between retraining cycles as the news stream changes. The widening should be fastest in low-resource languages, because they have the least feedback and the least pretraining coverage. The second is that a post-hoc repair layer can recover most of that lost parity at a small fraction of retraining cost. This layer sits on top of a frozen multilingual encoder and adjusts per-language thresholds from a mix of native and borrowed labels. The key ingredient is the borrowed labels. Per-language recalibration is an obvious idea and is our main ablation. Our claim is that without cross-lingual verdict transfer, recalibration has too few labels in long-tail languages to beat sampling noise.

Fairness here means equal error rates across language communities. The harm we care about most is the false-positive gap: legitimate reporting in a low-resource language being flagged as fake more often than comparable English reporting. In a moderation pipeline, that becomes suppressed speech for whole linguistic communities. We also track the false-negative gap, meaning hoaxes that go unflagged.

Where This Sits

Three bodies of work motivate the gap without filling it.

Cross-lingual misinformation benchmarks document the language gap, but only statically. BLUFF builds a 79-language benchmark of fact-checked and machine-generated content. It reports a 9.9 macro-F1 point gap between big-head and long-tail languages in cross-lingual transfer, which shrinks to 3.7 points with multilingual joint training. All of this is on fixed splits. SAFARI does media-level bias and factuality across 10 languages, with about 95% English data and static outlet labels. Cross-lingual COVID-19 Fake News Detection names the zombie-misinformation phenomenon and shows that translating Chinese into English beats native mBERT. byteSizedLLM shows that Romanized and code-mixed Malayalam needs script-aware modeling. None of these tracks performance over time or intervenes on fairness. We reuse their data and backbones, and we treat the zombie-claim observation as a source of labels rather than only as a threat.

Drift work in fake news supplies the evidence that time matters. Feature Drift in Fake News Detection shows that propagation features change their relationship with veracity across time windows and platforms. It explicitly offers no online mitigation, takes no fairness view, and covers English only. We extend the drift question to text-based detectors across languages, and we add mitigation.

Streaming fairness supplies the methods, but in the wrong setting. Continuous Fair SMOTE, FARF, FAHT and FEAMOE all adapt tabular models to drift under fairness constraints, and all assume labels arrive promptly with every example. CFSMOTE's authors also point out that the field lacks streams with real, rather than shuffled, drift. FAIR-SIGHT repairs a black-box vision model's outputs at inference time using a conformal threshold. It warns that its guarantees rely on calibration and test data being exchangeable, which is exactly what a drifting multilingual stream breaks. That warning constrains our design. We do not use split-conformal calibration. Instead we use online threshold tracking in the spirit of online multicalibration, which bounds error against an adaptive sequence rather than assuming exchangeability. The closest cross-lingual fairness-drift study, Cross-Lingual Fairness Drift in LLM Moral Reasoning, measures disparities on 50 static, translated dilemmas. It calls for sequential monitoring as future work and proposes no mitigation. We borrow its notion of a maximum cross-language gap as one reporting metric.

Research Program

The system has three parts on top of a frozen multilingual detector, either mDeBERTa or XLM-R fine-tuned on BLUFF-style training data.

The first part is a verdict-transfer module. Each incoming item is embedded with a multilingual sentence encoder and matched against a growing index of fact-checked claims in every language. A match above a similarity threshold, confirmed by a cross-lingual entailment check, inherits the source verdict with a confidence weight. The index grows as fact-checks are published, so transfer naturally runs from fast-feedback languages toward slow ones.

The second part is a per-language error monitor. For each language $\ell$ it estimates false-positive and false-negative rates on a sliding window, combining native and borrowed labels and shrinking toward the pooled estimate when evidence is thin:

$$\hat{e}_\ell = \frac{n_\ell \bar{e}^{\text{native}}_\ell + \lambda m_\ell \bar{e}^{\text{borrowed}}_\ell + \kappa \bar{e}^{\text{pool}}}{n_\ell + \lambda m_\ell + \kappa}.$$

Here $n_\ell$ and $m_\ell$ are the counts of native and borrowed labels in the window, $\bar{e}$ terms are the corresponding observed error rates, $\lambda \in [0,1]$ discounts borrowed labels for matching noise, and $\kappa$ sets how strongly sparse languages lean on the pool. Drift in $\hat{e}_\ell$ is flagged with ADWIN-style adaptive windows, as in CFSMOTE.

The third part is an online repair step. A per-language decision threshold $\tau_\ell$ moves after each labeled observation:

$$\tau_{\ell,t+1} = \tau_{\ell,t} + \eta\,(\hat{e}^{\text{FP}}_{\ell,t} - \alpha).$$

In this update, $\eta$ is a step size and $\alpha$ is a shared target false-positive rate. This is an online quantile-tracking update whose long-run error tracks $\alpha$ without exchangeability. The false-negative side is handled symmetrically, with a joint penalty on the gap to the best-served language.

The part we expect to be fragile is selection bias in borrowed labels. Claims that cross languages are probably more viral and more international than typical local news, so the monitor estimates errors on a skewed slice. We will measure this directly by comparing borrowed-label error estimates with held-out native labels where both exist. We will also test importance weighting on match-propensity features. Purely local claims never receive borrowed labels, and we will report how much of each language's stream falls into that blind spot.

The benchmark is itself a contribution, because no timestamped multilingual misinformation stream with real drift appears in the file. We will order fact-checked items by publication date using BLUFF's human-written fact-checked portion (122K+ samples, 57 languages), POLygraph ( POLygraph: Polish Fake News Dataset), MCFEND ( MCFEND: A Multi-source Benchmark Dataset for Chinese Fake News Detection), Ax-to-Grind Urdu ( Ax-to-Grind Urdu: Benchmark Dataset for Urdu Fake News Detection), FakeDetect-Malayalam, and the CrossFake Chinese set. Label arrival is delayed per language, with delays drawn from the gap between the original posting and the fact-check date where both are recorded. Where dates are missing, we fall back to explicitly simulated delays and report those results separately. We will not shuffle items, which is the practice CFSMOTE criticizes.

Baselines

The naive reference is the frozen detector with a single global threshold set once on the initial training data. This shows how much the language gap widens as the stream moves when nothing is done. If it does not widen, the first half of our claim fails.

The strongest existing method on the static question is multilingual joint fine-tuning as reported by BLUFF. There, joint training brings the big-head versus long-tail gap down to 3.7 macro-F1 points, and mDeBERTa reaches 67.3% F1 on external disinformation sources. We run it as periodic retraining at weekly and monthly intervals on all labels accumulated so far, using the same backbone and the same label stream including delays. We report its GPU-hour cost. We treat it as beaten if our repair layer matches its worst-language F1 and has a smaller time-integrated false-positive gap between retrains, at no more than a tenth of its compute. Retraining with access to borrowed labels is also run, so that any gain is not simply "more labels."

The strongest streaming fairness competitor is an adaptation of FEAMOE: new lightweight experts are added on top of frozen encoder features when drift is detected, with its demographic-parity and equalized-odds penalties applied across language groups. It gets the same labels and the same tuning budget. There is no reported result on Lacuna for any streaming fairness method on multilingual text. FAIR-SIGHT's over-30% reduction in parity gaps is for vision only, so we will establish this baseline ourselves.

The ablation that isolates our key ingredient is identical per-language threshold tracking that uses native labels only. We add two further controls: borrowed labels with random rather than matched verdicts, which tests for leakage, and shrinkage without borrowing. If native-only tracking matches the full system in long-tail languages, borrowing adds nothing and the proposal's central mechanism is refuted.

Why not just…?

Why not just retrain the joint multilingual model regularly? This is the real threat, because BLUFF's 3.7-point gap shows joint training works well when labels exist. Our argument is about the time between retrains and about label starvation. Retraining cannot fix a language for which no new labels arrived, and it costs full fine-tuning each cycle. The matched-label, matched-compute comparison above settles it. If weekly retraining closes the gap as fast as our layer at acceptable cost, the honest conclusion is that verdict transfer should feed retraining rather than replace it. We will report that combination too.

Why not just recalibrate thresholds per language on a sliding window? This is the native-only ablation. It should work for English and other big-head languages. Our prediction is that in long-tail languages the window holds too few labels for its error estimate to separate drift from noise, so it either reacts late or oscillates. If that prediction fails, we drop the borrowing machinery.

Why not just translate everything into English and use one strong English classifier? CrossFake supports this: it reports 75.00% accuracy against below-50% for mBERT on Chinese. byteSizedLLM shows it breaking on Romanized and code-mixed Malayalam, and CrossFake itself notes mistranslated emerging jargon. Translation also does not remove drift. It moves the drift into the translation step, where it can differ by language. We include translate-then-classify as a second backbone with and without our repair layer, which tests whether repair is backbone-agnostic.

Why not just prompt a large language model in each language? On BLUFF, zero-shot decoder models fell below random baselines on multiclass tasks, and SAFARI found zero-shot 7B models behind fine-tuned encoders. We include one prompted model as a reference row, but we do not expect it to be competitive.

Proposed Experiments

The first decisive check is whether drift is real and uneven. Using the frozen detector on the time-ordered stream, we plot per-language false-positive and false-negative rates over time. We report the maximum cross-language gap at each step, adapting the CPBD idea from Cross-Lingual Fairness Drift in LLM Moral Reasoning, and we compare growth rates between big-head and long-tail tiers. A flat gap would mean the problem is static, and existing static fixes suffice.

The second is whether borrowed labels are accurate enough. On languages with native fact-checks, we measure verdict-transfer precision and how far the borrowed-label error estimate is from the native-label error estimate, before and after propensity weighting.

The third is the main comparison against all baselines. Outcomes are worst-language macro-F1, the time-integrated false-positive and false-negative gaps, detection delay after an injected or natural drift event, per-language calibration error, and compute. Results are reported separately for FakeDetect-Malayalam's five-way labels, where the reported XLM-R plus Attention-BiLSTM system reaches 0.5775 macro-F1 with 'HALF TRUE' at 0.37 F1. This tests whether threshold repair helps intermediate veracity classes or only binary decisions.

The final check targets class imbalance. Because CFSMOTE shows that plain oversampling multiplies bias in streams, we test whether adding LLM-generated fake examples to long-tail languages widens the false-positive gap, both with and without the repair layer.