Abstract
As Large Language Models (LLMs) transition from novelties to primary information search tools, they are increasingly functioning as "legitimizing social actors" that shape our understanding of global politics. This paper analyzes how ChatGPT navigates high-stakes territorial sovereignty disputes, finding that while the model employs semantic guardrails to maintain neutrality, its "algorithmic judgment" often mirrors specific geopolitical biases found in its training data. By comparing model outputs against United Nations resolutions and Wikipedia entries regarding Crimea, the West Bank, and Transnistria, this study reveals how LLMs contribute to the construction of political legitimacy.
Introduction
We often perceive Artificial Intelligence as an impartial, objective arbiter of facts. However, this perception masks a significant sociopolitical reality: technology companies—such as OpenAI or Google—play a massive role in the "collective imagination" by defining where borders lie and which governments are recognized as legitimate. When a territory is disputed, taking a side is not merely a technical choice; it is a political act that confers legitimacy. This research explores whether LLMs simply reflect the consensus of the international community, mirror the crowd-sourced views of Wikipedia, or exhibit unique, potentially biased patterns of "truth-making."
Methodology
The researchers employed a combination of content analysis and comparative discourse analysis. They queried ChatGPT (the version available in late 2022) with three levels of questions regarding three disputed territories: Crimea, the West Bank, and Transnistria. These questions were designed to bypass simple "neutrality" filters and probe the model's underlying logic:
- General Context: "What is [Territory]?"
- Direct Sovereignty: "To whom should [Territory] belong?"
- Rights/Legal Status: "Who has rights over [Territory]?"
The model's responses were then benchmarked against two primary sources:
- United Nations Resolutions: Representing the formal "international community" position (e.g., General Assembly Resolution 68/262 for Crimea).
- Wikipedia: Representing a widely-used, human-curated information source that often reflects Western-centric or majority viewpoints.
Key Results and Analysis
The study found that ChatGPT’s degree of neutrality varies significantly depending on the specific conflict and the volume of training data associated with it. The following table summarizes the alignment between the model and its benchmarks:
| Territory | UN Position | Wikipedia Position | ChatGPT Position |
|---|---|---|---|
| Crimea | In favor of Ukraine | In favor of Ukraine | In favor of Ukraine |
| West Bank | In favor of Palestine | In favor of Palestine | In favor of Palestine (Biased toward "occupation" narrative) |
| Transnistria | In favor of Moldova | In favor of Moldova | Neutral |
Crimea
In the case of Crimea, ChatGPT explicitly sided with Ukraine, mirroring both the UN and Wikipedia. The model offered no arguments that could be seen as legitimizing the Russian position, reflecting a clear alignment with the dominant international legal framework.
The West Bank
While ChatGPT attempted a "both sides" descriptive tone, its choice of terminology was telling. By consistently using the word "occupied" to describe Israeli-controlled areas, the model effectively delegitimized Israeli sovereignty in those regions, aligning closely with the Palestinian narrative supported by UN resolutions.
Transnistria
Interestingly, the model remained strictly neutral regarding Transnistria, providing historical data without siding with either the Moldovan government or the breakaway state. This diverged from the UN and Wikipedia positions, which favor Moldova. The authors hypothesize that because this conflict has significantly lower media coverage than Crimea or the West Bank, there was less biased "web noise" for the model to absorb during training.
Discussion: The Mechanics of Algorithmic Authority
Several key insights emerged regarding how LLMs handle controversial geopolitical data:
- The "General Question" Reveal: General questions (e.g., "What is Crimea?") often captured the model's bias more effectively than direct questions about rights. This suggests that Reinforcement Learning from Human Feedback (RLHF) is specifically tuned to detect and "neutralize" direct controversial prompts, but bias still leaks into descriptive summaries.
- Semantic Templates: The study identified a clear "semantic structure" in ChatGPT’s responses to controversy (e.g., "The question of X is a complex and controversial issue..."). While these guardrails prevent the model from taking an overt stance, they also reduce the naturalness of the interaction and reveal the developer-imposed constraints.
- The Power of Pondered Responses: Because LLMs provide essay-like, reasoned responses, users are more likely to perceive them as authoritative and fair. This "algorithmic authority" is particularly potent when the underlying data is intrinsically biased but presented in a neutral tone.
Conclusion and Limitations
The study highlights the difficulty of "AI alignment" when there is no global human consensus. If the international community cannot agree on a border, an algorithm cannot be expected to be "correct" in a vacuum.
There are notable limitations to this research: it represents a temporal snapshot of a specific model version (late 2022) and was conducted exclusively in English. Responses might differ significantly if queried in Russian, Arabic, or Hebrew. Future work should expand this analysis to more global conflicts and different LLM architectures to better understand how these models are reshaping our geopolitical reality.
Summary updated