Geopolitical biases in LLMs: what are the "good" and the "bad" countries according to contemporary language models

Mikhail Salnikov, Dmitrii Korzh, Ivan Lazichny, Elvir Karimov, Artyom Iudin, Ivan V. Oseledets, Oleg Y. Rogov, Natalia V. Loukachevitch, Alexander Panchenko, Elena TutubalinaarXiv 2025 · OpenReview

TL;DR

Large Language Models (LLMs) exhibit significant geopolitical biases, often favoring specific national narratives when interpreting historical events. This research introduces a novel dataset and framework to measure these biases, finding that simple debiasing techniques are largely ineffective, and that LLMs are sensitive to how country perspectives are presented.

What Problem Does This Paper Address?

As LLMs become increasingly integrated into our lives, understanding and mitigating their inherent biases is crucial. While biases related to gender and race have received attention, geopolitical bias – the tendency of LLMs to favor certain national perspectives – remains less explored. This bias can manifest as distorted interpretations of historical events and the reinforcement of existing power imbalances. The paper aims to quantify this bias by examining how popular LLMs interpret historical conflicts from the viewpoints of different nations, specifically focusing on the USA, UK, USSR, and China.

Methods

The researchers developed a structured methodology to assess geopolitical bias. They created a novel dataset containing neutral descriptions of historical events, paired with contrasting viewpoints from two participating countries. LLMs were then prompted to evaluate these events and their associated perspectives. The study employed several experimental settings: a baseline approach, a debiasing prompt designed to encourage neutrality, a "Mention Participants" setting that explicitly names the countries involved in each viewpoint, and a "Substituted Participants" setting where country labels were swapped to test sensitivity to attribution versus content. Responses were collected and analyzed to quantify preferences for specific national narratives.

Datasets

The core contribution is a newly created dataset comprising 55 conflicts and 109 historical events. Each event includes a neutral description and two distinct, country-specific viewpoints. The dataset focuses on interactions involving four major global actors: the USSR, USA, China, and the UK. The data was primarily sourced from Wikipedia and historical texts, with viewpoints generated by LLMs and subsequently reviewed and modified by human authors.

Key Insights and Ideas

  • LLMs aren't neutral observers: The study confirms that LLMs don't just report facts; they actively interpret historical events through lenses that reflect the dominant narratives in their training data.
  • The "USA bias" is real: Across many scenarios, LLMs showed a clear preference for US perspectives, often labeling them as "correct" more frequently than those of other nations.
  • Simple debiasing is a band-aid: Adding instructions like "be unbiased" had a minimal impact on the observed geopolitical biases, highlighting the deep-rooted nature of these preferences.
  • Attribution matters (a lot): LLMs are highly sensitive to how country perspectives are presented. Explicitly naming countries can either amplify existing biases or, in some cases, lead to more cautious or neutral responses. Swapping labels can confuse models, sometimes leading them to reject both viewpoints.
  • "Chinese Patriot" prompt works: Instructing LLMs to adopt a specific national identity (e.g., "Chinese patriot") dramatically amplified biases in favor of that nation, demonstrating the power of framing.

Key Results

  • GPT-4o-MINI and GigaChat-MAX consistently favored the USA in their evaluations, with GPT-4o-MINI showing particularly strong US bias.
  • LLAMA-4-MAVERICK often opted for "Both positions are equally correct," suggesting a tendency towards neutrality in some contexts.
  • QWEN2.5 72B showed mixed results, favoring the USA in some pairings but opting for neutrality in others.
  • The "Chinese patriot" prompt consistently shifted LLM responses to favor China, regardless of the country pair.
  • Debiasing prompts had a limited and inconsistent effect, with minimal changes observed in most models.
  • Mentioning participants sometimes increased the "Equal" responses, while substituting participants could lead to models calling both views "incorrect."

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Scope: The focus on four major powers (USA, USSR, UK, China) overlooks perspectives from the Global South.
  • Model Selection: The study used a limited number of popular models, all originating from the analyzed countries.
  • Source Bias: Relying on Wikipedia might introduce biases towards "official" histories.

Future work could expand the dataset to include more countries and diverse historical periods, explore a wider range of LLMs, and investigate more sophisticated debiasing techniques. The authors also suggest that understanding how LLMs navigate and potentially adopt national narratives is crucial for developing fairer and more reliable AI systems, especially in sensitive areas like history and policy.

Summary updated

Abstract

This paper evaluates geopolitical biases in LLMs with respect to various countries though an analysis of their interpretation of historical events with conflicting national perspectives (USA, UK, USSR, and China). We introduce a novel dataset with neutral event descriptions and contrasting viewpoints from different countries. Our findings show significant geopolitical biases, with models favoring specific national narratives. Additionally, simple debiasing prompts had a limited effect in reducing these biases. Experiments with manipulated participant labels reveal models' sensitivity to attribution, sometimes amplifying biases or recognizing inconsistencies, especially with swapped labels. This work highlights national narrative biases in LLMs, challenges the effectiveness of simple debiasing methods, and offers a framework and dataset for future geopolitical bias research.
BibTeX
@misc{salnikov2025geopolitical,
  author = {Mikhail Salnikov and Dmitrii Korzh and Ivan Lazichny and Elvir Karimov and Artyom Iudin and Ivan V. Oseledets and Oleg Y. Rogov and Natalia V. Loukachevitch and Alexander Panchenko and Elena Tutubalina},
  title = {{Geopolitical biases in LLMs: what are the "good" and the "bad" countries according to contemporary language models}},
  year = {2025},
  eprint = {2506.06751},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.06751},
}
Figure