Cross-Lingual Factual Consistency in Large Language Models

This direction is split into other directions: cross-lingual knowledge inconsistency. Its last version stays readable and citable here.

Version 1 of 1201 papers359 conceptsfrozen 2026-10-04

Large Language Models (LLMs) are often marketed as "multilingual," yet they frequently exhibit a startling "consistency gap." A model might correctly identify a medical fact or a historical date when asked in English, but provide a contradictory answer or a hallucination when asked the same question in Hindi, Arabic, or Chinese. This phenomenon, known as cross-lingual factual inconsistency, reveals that a model’s internal knowledge is often siloed by language rather than stored as a universal, language-agnostic concept.

The Anatomy of the Consistency Gap

The core problem is not necessarily a lack of knowledge, but a failure of retrieval and alignment. As Ashim Gupta, Maitrey Mehta, and Zhichao Xu demonstrate in Found in Translation, models become significantly less informative and empathetic in non-Latin scripts. This isn't just a matter of translation quality; Nadezhda Chirkova and Vassilina Nikoulina found a distinct decoupling between linguistic fluency and factual accuracy. A model can be perfectly grammatical in a target language while simultaneously failing to retrieve the correct facts it clearly "knows" in English.

This disparity is particularly dangerous in high-stakes domains. In Better to Ask in English, Yiqiao Jin and colleagues show that healthcare queries in languages like Hindi often return contradictory or unverifiable advice compared to English counterparts. Similarly, Varun Gumma and Sunayana Sitaram identified a "performance tax" in multilingual retrieval-augmented generation (RAG) systems, where the ability to synthesize retrieved information into a correct response is not uniform across languages.

Why Knowledge Fails to Transfer

Research suggests three primary mechanisms behind these failures:

  1. Entity Misalignment: Models often fail to map different language-specific names for the same entity to a shared internal concept. Yihong Liu and Hinrich Schuetze argue in On the Entity-Level Alignment in Crosslingual Consistency that if a model cannot recognize that "London" and "Londres" refer to the same entity, it cannot share facts between those linguistic contexts. Masaki Sakata and Benjamin Heinzerling further show that models organize these entities into linear subspaces early in their layers, and poor organization here leads directly to factual recall failure.
  2. The Language Transition Bottleneck: Internal probing reveals that models often process facts in a language-neutral "concept space" in their middle layers but stumble during the final "language transition" phase. Mingyang Wang and co-authors in Lost in Multilinguality found that factual errors often occur when the model tries to map these internal concepts back to a specific target language.
  3. High Response Variance: The knowledge may be present in the model's parameters but is difficult to retrieve stably in non-English languages. Vihari Piratla and Purvam Jain suggest that the cross-lingual gap is often a retrieval issue; when multiple outputs are sampled and aggregated, the "hidden" correct answer often emerges.

Mending the Multilingual Pipeline

To bridge this gap, researchers are moving beyond simple fine-tuning toward structural and algorithmic interventions.

  • Internal Steering: Lu Meng and Carsten Eickhoff developed training-free vector interventions that steer the model’s internal pathways toward English-centric retrieval circuits, boosting accuracy in low-resource languages by over 35%. Similarly, Chenghao Xiao and Hou Pong Chan discovered that a model's "knowledge boundary"—its awareness of what it doesn't know—is encoded in a linear geometric structure that can be projected from English to other languages to reduce hallucinations.
  • Entity-Based Augmentation: Strategies like LEIA, proposed by Ikuya Yamada and Ryokan Ri, inject English entity names into target-language training data to "unlock" English-centric knowledge for other languages.
  • Strategic Alignment: Rather than using massive, noisy datasets, Weihao Liu and Ning Wu show in Selected Languages are All You Need that fine-tuning on a small subset of "core" languages can transfer truthfulness more effectively. Amit Agarwal and Sujith Ravi further suggest "batch-wise alignment," ensuring training batches contain semantically identical content across languages to force the model to build shared representations.

Ultimately, achieving cross-lingual consistency requires treating multilinguality not as a translation task, but as a challenge of knowledge synchronization. As Bryan Li argues, building models that are truly knowledgeable in a multilingual world requires moving past the "English-first" paradigm toward systems where facts remain invariant, regardless of the surface language used to query them.

Papers in this version

Showing 60 of 201 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_lingual_factual_consistency_in_lar_22e16b48,
  title = {Cross-Lingual Factual Consistency in Large Language Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-factual-consistency-in-large-language-models/rdr_22e16b48a15647f6afec44a57a29812a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_lingual_factual_consistency_in_lar_22e16b48_v1,
  title = {Cross-Lingual Factual Consistency in Large Language Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-factual-consistency-in-large-language-models/rdr_22e16b48a15647f6afec44a57a29812a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}