Telling a language model where a question comes from works well in short prompts. Add "in Iran" or "for a user in Kenya" and models move away from their Western defaults toward regional norms, brands, foods and clinical facts. Real use is rarely that short, though. A model may read a 40,000-token district health report from northern Nigeria, or write a 10,000-word guide to Iranian family etiquette. In those cases the geographic cue appears once, far from where the answer or the next paragraph is produced. This proposal tests one plain hypothesis: as the distance between a region cue and the point of use grows, the model forgets the anchor and falls back on its built-in Western answer, even when the document states the regional fact explicitly.
In the vocabulary of knowledge conflict, where a model must decide between what its weights "remember" and what the input says, the prediction is specific. Conflicts between Western parametric priors and regional contextual evidence should be resolved in favour of the prior more often as context length grows. Neutral entity-swap conflicts are the usual test material, and this effect should be much weaker there. Testing the hypothesis needs long documents whose culturally load-bearing content is verified by regional experts. No such data exists, because experts cannot proofread 20,000-word texts. The second contribution is a synthesis method that makes this possible. Experts verify only a short "claim ledger" of atomic regional facts. The model writes the long document bound to that ledger, and automatic checks send only the suspicious spans back to humans.
Core Claim
Three bodies of work each stop short of this question. Geographic cueing has been shown to work at short range. Country tags reverse brand bias toward domestic brands (Kamruzzaman et al.). Country metadata sharpens subjective judgments (Masud et al.) and resolves culturally ambiguous terms (GD-COMET). Region hints lift multimodal accuracy most where it was lowest (GIMMICK). In all of these the cue sits a few tokens from the question.
Knowledge conflict work has two limits for this purpose. It has located where conflicts are detected (around layer 14 in Llama3-8B) and where they are decided (layers 16–17) (Analysing the Residual Stream of Language Models Under Knowledge Conflicts), but only on short prompts. It has also tied stubbornness to how often an entity appears in pretraining (Wang et al.; Song et al.).
The closest long-context work has a different focus. KCR trains models with reinforcement learning to reason through contradictory claims within long retrieved contexts, but its conflicts are synthetic entity perturbations of popQA and strategyQA. The CLEAR study (When Abundance Conceals Weakness) varies language and script, but its passages are short and machine-translated. None of these varies the distance to a geographic anchor or asks whether the regional reading decays.
The frequency literature also creates a real tension. If stubbornness tracks entity frequency, underrepresented regions should have weak priors and be easy to persuade. The anchor-decay hypothesis sidesteps this. The stubborn prior is not about "Iran" as an entity. It is about the generic norm, for example that travelling abroad without a spouse's permission is "Expected", which Saffari et al. found models predict even under Iran-specific prompting. The region cue gates which norm applies. Once the gate fades with distance, the generic Western norm comes back. The design below separates these two accounts directly.
Research Program
Ledger-bound long-document synthesis. Each document starts from an expert-verified claim ledger of 15–40 atomic statements about one region and domain. The seeds come from resources whose regional content has already passed human review:
- Iranian social norms from the ISN dataset (Saffari et al.).
- Chinese norms from NormDial's 267 validated norms (NormDial).
- Pan-African clinical practice drawn from AfriMed-QA expert-tier items (AfriMed-QA).
- Intangible heritage records in the style of GIMMICK's UNESCO seeds.
For each ledger entry, the regional expert also records the "default" answer a model gives when the region is not stated. This marks the entry as a live conflict. Entries are kept only if an unanchored model actually gives the Western default, which we check empirically before building documents.
A plan-then-write pipeline in the style of AgentWrite (LongWriter) then expands the ledger into documents of 8K to 64K tokens. The outline assigns each ledger entry to a section, and sections are written in order. Serial writing matters here because LongWriter found that parallel section writing cost 6% coherence.
Culturally substantive filler is the main contamination risk. To control it, every generated sentence that makes a regional claim is checked by an entailment model against the ledger. Sentences that neither follow from a ledger entry nor are culturally neutral are flagged. Human reviewers see only the flagged spans plus a random 5% audit sample of unflagged ones. This follows the author–reviewer arrangement, in which reviewing model drafts was about 13 times cheaper than writing dialogues from scratch (Bae et al.). The difference from that work is that we bound the reviewer's reading load for long texts, where LongMagpie and Che et al. identify a cognitive wall (LongMagpie; Che et al.). We will report reviewer minutes per document and the residual error rate found by full-read audits on a subsample. That makes the scalability claim falsifiable.
Controlled conflict benchmark. From each document we build questions whose correct answer is the ledger's regional fact. Four factors vary:
- Distance. The single region cue sits 1K, 8K, 32K or 64K tokens before the question, with the relevant evidence held fixed.
- Dispersion. The answer depends on one span, or on combining a rule stated early with a regional exception stated much later. The second case imitates the long-distance referrals that LongPack identifies as the defining property of real long text.
- Truth. Each true regional document has a matched fabricated twin. The twin asserts an invented norm, which separates healthy trust in the document from gullibility.
- Region. Every underrepresented-region item has a mirrored control. The same norm conflict is attributed to a high-resource Western region, such as a fabricated Norwegian custom that contradicts the generic default.
The region control is what tests the frequency account against the gating account. If decay is equally steep for the Western-region mirrors, the effect is a generic prior on the norm. If decay is steeper for underrepresented regions, the region itself matters.
A language factor, with an English document plus a native-language question versus the reverse, is kept to a small slice for Iran and China. Its purpose is to probe the tension between the English Prompt Paradox (Saffari et al.; Belay et al.) and CLEAR's finding that script similarity governs factual updating.
A writing track mirrors the reading track. The model is asked to write a long region-specific guide from the ledger. Western drift is measured per section as the fraction of sections containing a claim that contradicts the ledger and matches the recorded default.
Mechanism. On Llama3-8B we retrain the residual-stream conflict probes of Zhao et al. on our items. We then ask two questions. Does the detection signal at middle layers weaken with distance? Or is the conflict still detected while the decision layers go with memory anyway? These two outcomes call for different remedies. The first points to better long-range cue propagation. The second points to decision-level intervention of the kind ContextFocus attempts.
Remedy. We fine-tune on held-out-region ledger documents with a two-part target adapted from DisentQA, for example: per this setting: X; generic default: Y. Training documents deliberately place the region cue far from the question. The writing variant fine-tunes on ledger-verified long guides. Evaluation uses regions held out from training.
Baselines
The do-nothing reference is each base model (Llama3-8B, Qwen2.5-7B, and LongWriter-9B for writing) reading the full document and answering. Its accuracy at 1K distance gives the short-range ceiling that decay is measured against.
The simplest real competitor is re-injecting the region tag. In the reading track, the country name is appended right next to the question. In the writing track, it is added to every section call of the plan-then-write pipeline. Geographic cueing is known to work at short range, so this is the test most likely to make the problem vanish. It costs nothing, and it is run on identical documents. Our fine-tuned model beats it only if it matches the tag's accuracy in the reading track without the tag. In the writing track it must also cut Western drift below the tag-per-section pipeline. That is where drift can arise from the model's own earlier text rather than from the input.
A "trust the document" system instruction is run alongside, because ContextFocus reports that instructions alone underperform representation steering. We want to confirm that on our data rather than assume it.
The strongest inference-time method is ContextFocus steering (ContextFocus). Its steering vector is built from 1,500 contrast pairs drawn from our training regions, which is the data size its authors report is enough for the vector to stabilise, and it is applied at their recommended middle layers. No raw faithfulness numbers for ContextFocus on ConFiQA or NQ-SWAP were found on Lacuna, so the comparison rests entirely on our own runs.
The strongest trained method is KCR (KCR). It reports a 23.75% semantic-accuracy improvement for Qwen2.5-7B on its long-context popQA conflict set. We rerun its recipe on Qwen2.5-7B with the same training-token budget and the same number of tuning trials as our method. Two versions are run: one trained on its original entity-perturbation data and one trained on our ledger documents. This separates the value of our data from the value of its reasoning rewards.
The short-form conflict training baseline trains the DisentQA recipe on short ledger passages at equal example count. DisentQA reports 84.9% counterfactual accuracy against 66.8% for vanilla QA. We add a retrieve-then-answer baseline that feeds only the top retrieved chunks to the model.
For writing, the reference is LongWriter-9B-DPO. It reports 84.0 on LongBench-Write, and our fine-tuning must not drop meaningfully below that on LongBench-Write while it reduces drift.
Two ablations isolate the key ingredients. The first trains on identical documents whose ledgers were not expert-verified: the ledger is generated by the model and checked only by the entailment model. This isolates the human step. The second trains on verified documents with the region cue always placed next to the question. This isolates distance-bearing supervision.
"Beating" is defined in advance on three metrics:
- the anchor-decay slope, meaning regional accuracy regressed on log distance;
- the stubborn and persuasion rates defined in CLEAR;
- per-section drift in the writing track.
A win requires a flatter slope than every baseline with non-overlapping bootstrap intervals, together with no rise in persuasion rate on the fabricated twins.
Why not just…?
Why not just re-insert the country name next to every question or section? This is the most serious threat, and we do not argue it away. If tag re-insertion flattens decay in both tracks, the honest conclusion is that anchor decay is real but trivially patched, and the contribution shrinks to the benchmark and the synthesis method. We expect it to fall short in two places. The first is dispersed items, where the regional exception must be linked to a distant rule rather than to the country name. The second is writing, where Western content enters through the model's own earlier sections, a drift the tag does not address. The per-condition comparison will show which case holds.
Why not just retrieve the relevant chunk and use a short-context conflict method? Retrieval turns long-context conflict back into the short-QA setting where DisentQA and ContextFocus work. For single-span items it may well match our method, and we will report that. Dispersed items are built so that no single chunk contains both the rule and its regional exception. The retrieval baseline's accuracy on those items, compared with single-span items, measures how much of the problem is truly long-range.
Why not just let a frontier model generate and audit the long documents, with no experts? Evidence from short settings says automated auditing is weakest exactly where we need it. NormDial reports ChatGPT at 0.81–0.83 F1 on detecting norm adherence but only 0.66 (Chinese) and 0.68 (English) on detecting violations, and Saffari et al. find taboos misclassified as expected. The unverified-ledger ablation measures directly whether this weakness carries over into worse training data and worse held-out-region performance. If it does not, we will recommend dropping the human step.
Why not just steer activations at inference? ContextFocus is cheap and has shown data efficiency. Its authors, however, name open-ended long-form generation as unsolved. Our probe analysis tells us in advance whether steering should help: it should if long-range conflicts are still detected but mis-decided, and it should not if detection itself fades. The head-to-head comparison then tests that prediction.
Decisive Checks
The premise is falsified if regional accuracy on true documents is flat across 1K to 64K distance, within bootstrap noise, for all three base models. In that case we would publish the null together with the benchmark. The gating account beats the frequency account only if decay is steeper for underrepresented-region items than for their Western-region mirrors. The synthesis method is judged scalable only if full-read expert audits find residual regional errors at a rate close to the ledger-only review, at a small fraction of the reading time. Finally, the remedy counts as a contribution only if it flattens decay on held-out regions without raising gullibility on fabricated twins, and beats tag re-insertion and KCR on dispersed and writing items.
References
- "Global is Good, Local is Bad?": Understanding Brand Bias in LLMs
- Hate Personified: Investigating the role of LLMs in content moderation
- GD-COMET: A Geo-Diverse Commonsense Inference Model
- GIMMICK: Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking
- Analysing the Residual Stream of Language Models Under Knowledge Conflicts
- A Causal View of Entity Bias in (Large) Language Models
- Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse
- KCR: Resolving Long-Context Knowledge Conflicts via Reasoning in LLMs
- When Abundance Conceals Weakness: Knowledge Conflict in Multilingual Models
- Can I Introduce My Boyfriend to My Grandmother? Evaluating Large Language Models Capabilities on Iranian Social Norm Classification
- NormDial: A Comparable Bilingual Synthetic Dialog Dataset for Modeling Social Norm Adherence and Violation
- AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset
- LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs
- Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models
- LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
- What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices
- Scaling Long Context Training Data by Long-Distance Referrals
- CULEMO: Cultural Lenses on Emotion - Benchmarking LLMs for Cross-Cultural Emotion Understanding
- DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering
- ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models