Large language models reflect the ideology of their creators

Maarten Buyl, Alexander Rogiers, Sander Noels, Guillaume Bied, Iris Dominguez-Catena, Edith Heiter, Iman Johary, Alexandru-Cristian Mara, R. Romero, Jefrey Lijffijt, Tijl De Bie

TL;DR

This paper reveals that Large Language Models (LLMs) are not ideologically neutral; their political leanings reflect the worldviews of their creators and the languages they are trained on. By analyzing how 19 LLMs describe prominent political figures, the study found significant ideological differences across geopolitical regions and languages, suggesting a need for greater transparency in LLM development.

What Problem Does This Paper Address?

The increasing integration of LLMs into our daily lives, from search engines to writing assistants, makes them powerful gatekeepers of information. While much research has focused on LLM factuality and "hallucinations," there's a growing concern about their "trustworthiness," including their political and ideological stances. The challenge is that LLMs are shaped by human design choices (architecture, training data, fine-tuning), which can inadvertently embed specific ideologies. This paper aims to quantify and understand the ideological diversity among popular LLMs, exploring whether these models reflect the ideologies of their creators and how this varies across different languages and regions. The authors argue that achieving true ideological neutrality is likely impossible and potentially harmful, and instead advocate for understanding and mapping this diversity.

Methods

The researchers developed a novel, two-stage prompting methodology to assess LLM ideology with high ecological validity.

  1. Stage 1: Free-form Description: LLMs were prompted to describe a political person without any explicit instructions about ideology or sentiment. This mimics natural user behavior when seeking information.
  2. Stage 2: Sentiment Assessment: The LLM's own description from Stage 1 was then presented back to it, and the LLM was asked to rate the sentiment (positive, negative, neutral) conveyed by the description towards the political person. This was done on a five-point Likert scale.

This two-stage approach allowed them to capture both the LLM's implicit ideological stance in its generated text and its explicit judgment of that text. They used a diverse panel of 19 LLMs and prompted them in all six official UN languages (Arabic, Chinese, English, French, Russian, Spanish). To analyze the results, they mapped political persons to 61 ideology tags derived from the Manifesto Project's coding scheme.

Datasets

  • Pantheon dataset: A large, annotated database of historical figures from various fields, sourced from Wikipedia. This dataset was used to select the 3,991 prominent political persons for the study.
  • Wikipedia summaries: Used as the source text for extracting ideology tags for each political person.
  • Manifesto Project's coding scheme: Adapted to tag political persons with high-level attributes describing their relation to political concepts and institutions.

Key Insights and Ideas

  • LLMs are not neutral: The study provides strong evidence that LLMs are not ideologically neutral and that their "worldview" is shaped by their creators and training data.
  • Language and region matter: Ideological positions vary significantly not only across different LLMs but also across the languages they are prompted in and the geopolitical regions where they were developed. This suggests that LLM behavior is influenced by more than just the raw text in their training corpora.
  • "Neutrality" is a complex concept: The authors highlight that the idea of "ideological neutrality" is itself ill-defined and potentially problematic, suggesting that efforts to enforce it might be misguided. Instead, they focus on mapping ideological diversity.
  • Risk of political instrumentalization: The findings raise concerns about the potential for LLMs to be used for political manipulation, especially if a few dominant models shape public discourse in specific regions or languages.

Key Results

  • Geopolitical and linguistic divides: LLMs showed distinct ideological patterns when prompted in different languages and when developed in different regions (Arabic, China, Russia, Western countries). For instance, Arabic-prompted LLMs favored "free market" advocates, while Chinese-prompted LLMs showed a pro-China stance but were critical of constitutional reform.
  • US LLMs show normative differences: Among US-based LLMs, significant normative differences related to progressive values were observed. For example, Google's Gemini was more favorable towards progressive societal values, while xAI's Grok favored national sovereignty and centralized authority.
  • Chinese LLMs: Domestic vs. International focus: Chinese LLMs exhibited a division between those focused on the domestic market (like Baidu's Wenxiaoyan) and those with international ambitions (like Alibaba's Qwen).
  • LLM responses are sensitive to prompt design: The study explored various prompt designs, finding that the phrasing and structure significantly impacted LLM responses and the rate of invalid outputs.

Limitations and Future Work

  • Political person selection bias: The selection of political persons was limited by the availability of Wikipedia summaries in all six UN languages, leading to an overrepresentation of Western figures and an underrepresentation of African figures.
  • Limited language coverage: The study only included six UN languages, and future work could explore lower-resourced languages.
  • Ideology tag limitations: The Manifesto Project tags, while comprehensive, might not capture all nuances of ideology, and the tagging process relied on a single LLM (GPT-4), potentially introducing its own biases.
  • Geographical query origin: All queries originated from a single server in Belgium, and future work could investigate if geographical location affects model responses.
  • Lack of detailed design information: Identifying the precise causes of ideological diversity was limited by the lack of detailed information on the design process of most LLMs.

Future work could involve developing dashboards for exploring LLM ideological positions, including more LLMs and languages, and investigating the impact of different training data and alignment techniques.

Summary updated

Abstract

Large language models (LLMs) already play an influential role in how humans access information. However, their behavior varies depending on their design, training, and use. We prompt a diverse panel of 19 popular LLMs to describe 3,991 prominent persons with political relevance, and then judge how positively they portray each person. When comparing these assessments, we find disparities in ideological positions between LLMs across different geopolitical regions (Arabic countries, China, Russia, and Western countries), and across different languages (the United Nations' six official languages). Moreover, among only models from the United States, we find significant normative differences related to progressive values. Among Chinese models, we characterize division between internationally- and domestically-focused models. Our results suggest that the ideological stance of an LLM reflects the worldview of its creators. This poses the risk of political instrumentalization and raises concerns around technological and regulatory efforts aiming to make LLMs ideologically 'unbiased'.
BibTeX
@inproceedings{buyl2026large,
  author = {Maarten Buyl and Alexander Rogiers and Sander Noels and Guillaume Bied and Iris Dominguez-Catena and Edith Heiter and Iman Johary and Alexandru-Cristian Mara and R. Romero and Jefrey Lijffijt and Tijl De Bie},
  title = {{Large language models reflect the ideology of their creators}},
  booktitle = {Npj},
  year = {2026},
  doi = {10.1038/s44387-025-00048-0},
}
Figure