Blaming the Bridge in Cross-Lingual Fact-Checking

When a fact-checking system verifies a Hindi claim against English evidence, or a Spanish claim against Chinese news, something has to carry meaning across languages. That something might be a machine translator, a word aligner, or a shared multilingual embedding. This "bridge" makes mistakes. It drops a negation, turns "partly" into "fully," or swaps one entity for another. Explainable fact-checkers try to show which part of a claim is false. They are built and tested on English, so they have no way to tell a false claim apart from a faithful claim that was garbled in transit. The likely result is the worst kind of explanation: a fluent, confident justification that blames the wrong phrase of a true claim, citing evidence that says the opposite in the original language.

We propose to make the bridge a first-class suspect inside the explanation. The verifier checks each phrase of a claim against evidence through several independent bridges. It also runs a translation-error detector on the aligned evidence span. It then assigns blame either to the claim phrase or to the bridge, and it refutes a claim only when the refutation survives a reliable crossing. The contribution is measured as much by explanation correctness as by verdict accuracy. The key quantity is how often a system's stated culprit is actually a translation error. No existing benchmark measures this, so we build one from human-annotated cross-lingual divergence data.

Core Claim

Our central claim is that cross-lingual explainable fact-checking has a distinct failure mode, which we call bridge misattribution: an alignment or translation error is reported to the user as a falsehood in the claim. We further claim this can be reduced substantially by extending phrase-level culprit localization with an explicit bridge-reliability variable. Verdict accuracy should stay unchanged or improve slightly. The gain should be concentrated on typologically distant pairs such as English–Hindi.

The cross-lingual alignment literature motivates the claim. Translation errors split into omissions and substitutions, and substitutions mislead readers more (Revisit Automatic Error Detection for Wrong and Missing Translation). Noise in the bridge passes straight into any labels or structures projected across it (Mihalcea et al.). Downstream zero-shot relation classification is bounded by mapping quality (Glavaš et al.). Kociský et al. showed that representations stay coherent only when alignment uncertainty is marginalized rather than treated as fixed. That is the methodological hint we take: treat the bridge as uncertain, not as ground truth.

The method we build on comes from explainable fact-checking. LOREN splits a claim into phrases. It gives each phrase a three-valued hidden verdict (supported, refuted, not enough information) and trains with a soft-logic rule: a claim is refuted if any phrase is refuted and supported only if all phrases are. With that rule it finds culprit phrases without phrase-level labels. LOREN is entirely monolingual and has no notion that evidence could be distorted before it reaches the model; we add exactly that.

The closest cross-lingual systems leave the gap open. CONCRETE retrieves evidence across languages but ends in an opaque seven-way classifier that cannot say why. X-PARADE annotates which spans of a paragraph are the same as, inferable from, or new relative to a paragraph in another language. It is a diagnostic benchmark only, with no verification system attached.

Research Program

Verifier. The system takes a claim in language A and evidence in language B. Following LOREN, it splits the claim into phrases. For each phrase $p$ it computes a phrase verdict through $K$ bridges:

  • a direct cross-lingual reader that sees the original-language evidence;
  • two or more distinct machine translation systems, each followed by an English phrase verifier;
  • a word-alignment route that locates the evidence span the phrase depends on.

A translation-error detector inspects that aligned span for omissions and substitutions. It is trained in the style of the supervised detector of Lei et al. on their Chinese–English TransErr data and on synthetic corruptions in our other languages. The detector's output, together with disagreement among the bridges, gives a reliability score $\rho_p \in [0,1]$ for phrase $p$'s evidence crossing.

Aggregation rule. Let $q_p^R$ be the pooled probability that phrase $p$ is refuted. The claim-level refutation probability counts only refutations that crossed a reliable bridge:

$$P(\text{refuted}) = 1 - \prod_{p} \left(1 - \rho_p \, q_p^R\right).$$

The probability mass that is refuted only through unreliable bridges goes to a separate outcome, "unresolved: translation." This outcome is shown to the user together with the original-language span and the disagreeing translations. At the verdict level it is scored as not enough information. As in LOREN, a soft-logic teacher distills this rule into the phrase-level model. Synthetic corruptions, where we know which span was damaged, supply weak supervision for $\rho_p$.

Test set. The second component is a test set where blame is known. We start from X-PARADE paragraph pairs (ES→EN, ZH→EN, EN–HI). Annotators, assisted by a language model and verified by humans, write claims grounded in the English paragraph, with the other-language paragraph serving as evidence:

  • Claims from spans labeled Same or Inferable are Supported.
  • Claims from spans labeled New are not-enough-information.
  • Minimal edits to Same spans (negations, entity swaps, scalar changes) produce Refuted claims.

On top of this we create two evidence conditions:

  • Natural: evidence passed through real machine translation systems. A sample is annotated for whether each wrong verdict traces to a translation error.
  • Controlled: we inject omission and substitution errors of the TransErr types into the translated evidence. This yields exact labels for when the bridge, not the claim, is the culprit.

The first study uses this resource before any modeling: how often do existing explainable verifiers misattribute bridge errors on naturally translated evidence? If that rate is negligible, the problem is minor, and we will report that rather than press on.

Baselines

Naive cascade. The do-nothing reference translates the evidence into English with a single strong machine translation system and runs LOREN unchanged. This is what a practitioner would deploy today. It isolates the cost of treating the bridge as transparent. We use the same phrase decomposer, the same verifier backbone, and the same tuning budget as our system.

Prompted multilingual LLM. The strongest existing approach on our test set is a frontier multilingual model prompted with chain-of-thought. It reads the original-language evidence directly and must name a culprit phrase or flag a translation issue. The best reported cross-lingual span-divergence result for such a model is GPT-4's New-information detection F1 of 79.3 on ES→EN in X-PARADE. The same paper reports that model scores on Inferable spans fall 10–20 points below human agreement. Because this model never sees a translation, it is the natural competitor to our direct-reader bridge. To beat it, we must match its verdict accuracy and achieve lower misattribution, especially on EN–HI.

End task on real claims. We use CONCRETE's retriever and compare against its reported zero-shot macro F1 of 19.83% on X-FACT. X-FACT's own best in-domain system reaches 41.9% macro F1 with evidence and metadata, and a claim-only model reaches 38.2%. Both are reported by X-Fact. On this task we aim only for no loss in verdict accuracy, and we evaluate explanations by human audit.

Ablations. Two ablations isolate the key ingredient:

  • Majority vote: keep the $K$ bridges but replace $\rho_p$ with a plain vote over bridge verdicts. This tests whether the bridge variable adds anything beyond ensembling.
  • Single bridge: keep $\rho_p$ but use one bridge plus the error detector. This tests whether disagreement among bridges, or detection alone, carries the signal.

All systems use gold evidence on the test set, so retrieval does not confound the comparison. They share the same translation systems and backbone, with equal hyperparameter search budgets.

Why not just…?

Why not just translate everything into English and run a strong English verifier? This is the naive cascade above. Its failure mode is exactly what we measure: an omitted negation reaches LOREN as clean evidence and produces a confident, wrong culprit. Lei et al. document that such omissions and substitutions are common. X-Fact found that adding English fact-checking data hurt multilingual performance, which is a warning against assuming English pipelines transfer. The controlled condition settles the question directly, because there the true culprit is known.

Why not just prompt a frontier multilingual model on the original evidence? This is a real threat, since it bypasses translation entirely. Two pieces of evidence count against it. Its documented weakness on Inferable content and on Hindi (X-PARADE) suggests it will miss or invent divergences in exactly the hard cases. Free-form rationales are also not tied to the decision, a concern raised by LOREN and by SEVA, which needed process rewards to get reliable structured attributions. If the prompted model matches our misattribution rate on EN–HI, we will conclude that the direct reader alone suffices and the multi-bridge machinery is unnecessary.

Why not just filter evidence with a translation-error detector, or vote across several translations? Either fix could recover most of the verdict accuracy. Both are therefore included as ablations, not dismissed. Each has a specific weakness:

  • Filtering discards evidence. On a hard case, the Missing-word detector of Lei et al. reaches only 0.400 F1, so it will both miss errors and throw away good spans.
  • Voting gives a verdict but cannot tell the user that the disagreement came from translation.

Our claim survives only if the bridge variable lowers misattribution relative to both. If voting matches it, the honest conclusion is that ensembling is enough. The remaining contribution would then be the test set and the misattribution metric.

Proposed Experiments

Prevalence study. On the naturally translated condition, we measure how often the naive cascade, the prompted LLM, and a QA-NatVer-style natural-logic verifier produce wrong culprits that trace to the bridge, broken down by language pair. This establishes whether the problem matters.

Main comparison. On the controlled condition, we report three numbers for each system:

  • verdict accuracy;
  • culprit F1, extended so that "bridge" counts as a valid culprit (LOREN's monolingual culprit F1 of 74.3 serves as a reference for what this metric looks like);
  • the bridge misattribution rate.

The decisive result is a large drop in misattribution relative to the voting ablation, with verdict accuracy within noise of the best baseline, and with the gap widening from Spanish to Hindi. If misattribution falls only as much as it does under voting, the central claim fails.

Real claims. On X-FACT with CONCRETE retrieval, we check that macro F1 does not fall below the reported 19.83% zero-shot figure. Professional or trained annotators then audit 200 explanations per system and record whether each stated reason is correct in the original language.

User study. Annotators judge whether a verdict is right, given either our explanations or LOREN-style explanations. The measure is how often they catch bridge-induced errors. Atanasova et al. found that better explanations helped humans recover the gold label, which makes this the right end-user test.