Fact-checkers working in languages other than English need verdicts they can audit: which part of a claim is wrong, and which part of the evidence shows it. Explainable fact-checking systems have mostly been built and trained on English. Multilingual systems mostly produce opaque labels. We propose to train a fact-verification model on English only and have it write its explanation as a short proof. Each proof step points at a span of the claim and a span of the evidence, then names a logical relation between them, such as "equivalent," "contradicts," or "more specific than." The proof contains no words in any human language. The model writes only position pointers and a small fixed set of relation symbols, and a grammar enforced during decoding guarantees that every proof is well-formed and that the verdict follows mechanically from it. Because the output vocabulary is the same in every language, the model never has to generate Danish, Swahili or Urdu text. We expect this to remove a major source of failure when structured outputs are transferred to unseen languages. It should also keep the explanation short and tie it directly to the original-language text.
The bet rests on two observations from the literature. First, writing structure out as text is expensive and fragile. When trees are flattened into bracketed strings, sequence length roughly doubles. Low-resource training suffers from the extra tokens. Inserting markers into text corrupts multilingual translation models. Second, constraining a decoder to a formal grammar reliably fixes malformed output in small models, but it can hurt large ones. We therefore combine the cheapest possible proof encoding with the smallest possible grammar and test zero-shot transfer from English to native Danish data and to several lower-resource languages.
Core Claim
The claim is that the hard part of zero-shot cross-lingual explainable verification is not deciding the verdict. It is producing the structured explanation in a language the model was never trained to generate. If the explanation is written as pointers into the input plus logical operators, and the decoder is constrained to a per-instance grammar, three things should follow relative to proofs that copy span text. Proof validity should stay near 100% in every language. Proof-level accuracy, meaning correct culprit spans, should drop much less from English to unseen languages. Output length should become independent of how many subword pieces the target language needs per word. The verdict is faithful by construction, because it is read off the proof by a finite automaton rather than predicted separately.
The building blocks come from three places.
Natural-logic proofs. QA-NatVer casts operator assignment as boolean questions to instruction-tuned models and routes the operators through a deterministic finite automaton to produce a verdict. It reaches 70.3% FEVER accuracy with Flan-T5. It is an English system that queries every claim–evidence chunk pair without constraints, and it was only tested on Danish. We keep its automaton and operator inventory. We replace exhaustive pairwise querying with a single constrained generation pass.
Phrase-level culprits. LOREN gives phrase-level truth values combined by logic rules and reports 74.3 F1 on locating the false "culprit" phrase. It relies on English constituency parsing. We need no target-language parser, because the grammar lets the model choose segment boundaries at word boundaries.
Grammar-constrained decoding, meaning masking at each step every next token that would break a formal grammar, comes from Grammar-Constrained Decoding Makes Large Language Models Better Logical Parsers. That paper both supplies the method and constrains it. Constraints lifted gemma2-2b's executable rate from near zero to over 60%. They also dropped qwen2.5-14b on GSM-symbolic from 0.56 to 0.36 in the 5-shot setting, so scale must be an explicit experimental axis.
Research Program
Proof language. A proof is a sequence of steps. Each step is written as ⟨c_i, c_j, e_k, e_l, op⟩:
c_i, c_jare word positions in the claim, marking the start and end of a claim span.e_k, e_lare word positions in the evidence, or a null pointer for "no matching evidence."opis one of the natural-logic operators.
The per-instance grammar enforces three rules:
- Claim spans must tile the claim left to right, with no gaps or overlaps.
- Evidence pointers must fall inside the given evidence.
- The operator sequence must be a path through the QA-NatVer automaton, whose final state is the verdict.
This grammar is regular and tiny. At each step only a handful of positions or operators are legal. That should make decoding cheap and leave the model fewer opportunities to be distorted by the constraint.
Pointer mechanism. In the main system, a multilingual encoder–decoder (mT5 or mBART-50) gets a pointer head that attends over encoder word positions, so no index markers are added to the input. A second variant uses a decoder-only multilingual instruction model. Here we insert position markers into the input only at word boundaries. We measure the resulting input expansion directly, because this is exactly the length cost described by Towards String-To-Tree Neural Machine Translation and UniXcoder.
Training data. We run QA-NatVer on English FEVER and keep only proofs whose automaton verdict matches the gold label. We convert their chunks to word pointers and train on these silver proofs. Filtering on the label does not eliminate spurious proofs, which reach the right answer by wrong reasoning. Learning to Generate Programs for Table Fact Verification via Structure-Aware Semantic Parsing found that weakly supervised program generators matched gold logic on only 15.6% of Logic2Text instances despite good label accuracy. For that reason we evaluate proof correctness against human annotation, not label accuracy alone.
Scope. The program studies verification given evidence. We use gold FEVER evidence and native DanFEVER evidence so that reasoning transfer is isolated from retrieval. The FEVER baseline fell from 50.91% label accuracy ignoring evidence to 31.87% FEVER score when correct evidence had to be retrieved, so retrieval is a separate, already-studied bottleneck.
Baselines
Naive reference points. These are the majority class and a claim-only multilingual classifier fine-tuned on English FEVER and applied zero-shot. They tell us how much the evidence and the proof structure contribute at all.
Strongest reasoning baseline: translate-test QA-NatVer. Target-language claim and evidence are machine-translated to English with NLLB. QA-NatVer with Flan-T5 runs on the translation. Its spans are projected back to the original text with the two-pass constrained projection of Constrained Decoding for Cross-lingual Label Projection. That method translates cleanly, then forces the model to reproduce the translation while choosing where span boundaries go; it reached 70.4 average F1 on MasakhaNER 2.0. This is the most serious competitor because it reuses the best English explainer plus the best available projection method. Its reported English number is 70.3% FEVER accuracy. For DanFEVER, QA-NatVer is evaluated but no reported result was found on Lacuna for its Danish accuracy. We will rerun it ourselves with the same Flan-T5 size and the same few-shot examples.
Supervised reference ceiling. XLM-RoBERTa Large trained in-language on DanFEVER as entailment reaches 90.2 F1 (DanFEVER). This is a supervised target-language ceiling, not a zero-shot competitor. We report it so readers can see the remaining gap.
Fairness. All learned systems get the same English FEVER training claims, the same backbone family and size where applicable, and the same hyperparameter search budget, tuned on English dev only. No target-language tuning is allowed.
Ablation of the key ingredient. Our method beats the copy-text ablation if it improves target-language culprit-span F1 by a margin outside bootstrap confidence intervals while matching label accuracy. It beats translate-test if it matches translate-test's label accuracy on the lower-resource languages, wins on culprit-span F1 measured in original-language text, and needs fewer generated tokens and no MT calls. The ablations are:
- Copy-text ablation. The identical model and grammar, but spans are written out as copied text inside brackets. This isolates whether pointers, rather than constraints, drive transfer.
- Unconstrained pointer variant. Pointers with no grammar mask. This isolates the constraint.
- Scale sweep. Both variants on decoder-only backbones from about 2B to 14B parameters, to locate any inversion like the one in the logical-parser paper.
Why not just…?
Why not just translate everything into English and run QA-NatVer? This is a real threat. On Danish, where MT is strong, it may match us on label accuracy. Prior evidence suggests pipelines compound errors. In Cross-lingual Semantic Parsing, translate-then-parse scored 23.95 F1 against 38.38 for a direct model. Translation also degraded reading comprehension relative to direct multilingual transfer in Zero-shot Reading Comprehension by Cross-lingual Transfer Learning. Translate-test also produces explanations about the English translation, which must then be projected back, and projection itself is an error source. We settle this per language and report it honestly. If translate-test wins on accuracy everywhere, the remaining contribution is cost and native-language grounding, and the note will say so.
Why not just prompt a large multilingual model to explain its verdict in free text or JSON? Free-text explanations are not faithful by construction: the verdict and the explanation are generated separately. For small models, unconstrained structured output is nearly never executable, as shown by gemma2-2b's near-zero first-order-logic executable rate in the logical-parser paper. For large models, the same paper shows unconstrained 5-shot prompting can beat constraints. We therefore include a prompted unconstrained 14B baseline that produces the same pointer proofs, and we report where on the scale curve the constraint stops helping.
Why not just generate, check, and resample invalid proofs instead of masking? Rejection sampling also guarantees validity eventually. Its cost grows with the invalid rate, which we expect to be highest in exactly the unseen languages. We measure both validity-after-k-samples and total decoding tokens against masked decoding. If resampling reaches equal accuracy at similar cost for large models, we will recommend it there.
Proposed Experiments
Training and in-distribution evaluation. Training uses English FEVER only. In-distribution evaluation is on the FEVER dev set with gold evidence.
Zero-shot transfer. Native transfer is tested on DanFEVER: 6,407 claims, 3-class, with native evidence. To reach lower-resource languages, we translate a 2,000-claim FEVER dev sample into Turkish, Hausa, Urdu, Swahili and Vietnamese with NLLB and human-check a 200-claim subset per language. We flag that translate-test then sees doubly translated text, which may advantage or penalize it.
Real-world stress test. We also run the X-Fact zero-shot split, mapping its seven labels onto three. Prior zero-shot macro F1 there is 15–16%, and CONCRETE reports 19.83%. We expect natural logic to be weak on these numerical and political claims and treat this as a limit test rather than a headline.
Metrics.
- Label accuracy and macro F1.
- Proof validity rate.
- Culprit-span F1 against 200 human-annotated refuted claims per language. These are annotated in the original language following LOREN's protocol.
- Generated tokens per proof and wall-clock decoding time.
Decisive checks.
- Transfer gap. Is the English-to-target drop in culprit-span F1 smaller for pointer proofs than for copy-text proofs, and does the advantage grow for languages whose tokenizer splits words into more pieces? The second part is a hypothesis we will measure, not assume.
- Scale. Does masking stop helping, or start hurting, above some model size?
- Low-resource head-to-head. Does the pointer system match translate-test label accuracy on Hausa and Urdu while beating it on original-language culprit spans?
If the copy-text ablation transfers as well as pointers, the central claim is wrong.
References
- QA-NatVer: Question Answering for Natural Logic-based Fact Verification
- LOREN: Logic-Regularized Reasoning for Interpretable Fact Verification
- Grammar-Constrained Decoding Makes Large Language Models Better Logical Parsers
- Towards String-To-Tree Neural Machine Translation
- UniXcoder: Unified Cross-Modal Pre-training for Code Representation
- Learning to Generate Programs for Table Fact Verification via Structure-Aware Semantic Parsing
- FEVER: a Large-scale Dataset for Fact Extraction and VERification
- Constrained Decoding for Cross-lingual Label Projection
- DanFEVER: claim verification dataset for Danish
- Cross-lingual Semantic Parsing
- Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model
- X-Fact: A New Benchmark Dataset for Multilingual Fact Checking
- CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval