Abstract
The confusion around the term “Artificial General Intelligence” (AGI), often trapped and disputed between the marketing and research fields, deserves to be defined and analyzed from an ethical perspective. In 1980, American philosopher John Searle published an article in which he argued against what was then called “strong AI.” Following the legacy of Alan Turing, the question Searle posed was: “Is a machine capable of thinking?” (Searle, 1980). To briefly summarize the experiment, the philosopher illustrated a thought experiment known today as “the Chinese room” to attempt to answer his question. The thought experiment consists of imagining a room in which Artificial Intelligence (AI) has at its disposal a set of documents (knowledge base) with Chinese sentences in it. A native Chinese speaker enters the room and begins to converse with this AI; the latter can answer, considering it can easily find which sentence corresponds to the questions asked. The American philosopher’s argument is simple: although AI can provide answers in Chinese, it has no background knowledge of the language. In other words, the syntax is not a sufficient condition for the determination of semantics.
Introduction
The transition from specialized artificial intelligence to "general-purpose" Large Language Models (LLMs) represents more than a technical leap; it marks a significant shift in the ethical landscape of technology. As models like GPT-3 and its successors are marketed as precursors to Artificial General Intelligence (AGI), they introduce systemic risks that challenge our ability to govern and assess AI impact. This paper explores the "moral landscape" of these models, arguing that the pursuit of universality often comes at the cost of labor ethics, linguistic diversity, and safety.
The Ambiguity of General-Purpose AI
Historically, AI systems were designed for specific, bounded tasks—such as playing chess or diagnosing medical images. This specificity allowed for clear "ex-ante" (before the fact) moral assessments. We could predict risks because the domain was known. In contrast, general-purpose LLMs are designed to be "open-ended." Their capabilities are not defined at the time of creation but emerge through user interaction.
This creates what Giada Pistilli identifies as the Capability Trap. When a tool can be used for anything—from creative writing to generating sophisticated phishing scripts—it becomes nearly impossible to implement comprehensive safeguards. The "un-governability" of these models stems from this inherent unpredictability.
Philosophical Foundations: Syntax vs. Semantics
To understand the limitations of LLMs, we must revisit John Searle’s "Chinese Room" thought experiment. Searle argued that a machine could manipulate symbols (syntax) to provide correct answers without ever understanding the meaning (semantics) of those symbols. Modern LLMs are the ultimate realization of the Chinese Room. They are probabilistic engines that predict the next token based on statistical patterns.
The confusion between syntax and semantics leads to the "Sentient AI Trap," where users attribute human-like understanding or morality to a system that is essentially a sophisticated calculator. This misunderstanding complicates safety efforts, as we attempt to apply deterministic "guardrails" (like content filters) to systems that are fundamentally probabilistic. Because LLMs operate on probability, there is always a non-zero chance of generating toxic or harmful output, regardless of the filters in place.
The Technoeconomy of AI Labor
The development of "general-purpose" AI relies on a hidden "technoeconomy" of human labor. To make these models safe for public consumption, companies employ "data-labeler proletarians," often located in the Global South. These workers are tasked with reviewing and labeling thousands of snippets of toxic content—including descriptions of violence and abuse—to train the model's safety filters.
Reports have documented workers in countries like Kenya earning less than $2 per hour for this psychologically taxing work. This creates a moral paradox: the effort to make AI "ethical" for Western consumers often relies on the exploitation of vulnerable labor markets. This "dark process" is a foundational component of the current AI industry that is frequently omitted from the marketing of AGI.
Linguistic Monoculture and the Myth of Universality
While AGI is often framed as a universal benefit to humanity, the data tells a different story. Training sets are overwhelmingly dominated by English and Western cultural values. For instance, approximately 93% of GPT-3’s training data was in English, while languages like French, German, and Spanish each accounted for less than 2%. This creates a "linguistic monoculture" that marginalizes minority languages and flattens global cultural diversity.
When AI models are exported globally, they carry the biases and social norms of their training data. This lack of pluralism means that minority languages, such as Catalan or various African languages, are either poorly served or forced to conform to an Americanized linguistic framework. Research indicates that GPT-3 performs significantly worse in these low-resource languages, reinforcing digital divides.
Conclusion: Toward a Pluralistic Future
The pursuit of a single, all-encompassing AGI may be fundamentally at odds with ethical AI development. Instead of striving for "general-purpose" models that are difficult to regulate and culturally homogenizing, we should consider a pivot toward Narrow AI.
Narrow AI involves models tailored to specific contexts, languages, and communities. By focusing on decentralized, grassroots initiatives—such as the Masakhane project for African languages—we can promote data sovereignty and ensure that AI reflects the plurality of human values. Moving forward, the collaboration between engineers and philosophers must prioritize "Human-Centered AI," ensuring that technical progress does not outpace our moral responsibility to the workers and cultures that sustain it.
Summary updated