This position paper challenges the prevailing trend of treating Artificial Intelligence as an "ethical agent" capable of human-like moral reasoning. By drawing on political philosophy, the author argues that our current anthropomorphic approach to AI ethics is a conceptual dead-end. Instead, we should view AI systems as the outcomes of complex political processes.
The Anthropomorphic Trap in AI Evaluation
In recent years, the AI research community has increasingly turned to moral psychology to evaluate Large Language Models (LLMs). We see benchmarks asking if GPT-4 is "more moral than a graduate student," or using datasets like Reddit's "Am I the Asshole" (AITA) and the Defining Issues Test to measure a model's "moral development."
The fundamental problem with this approach is that it assumes AI can be an "agent" in the same way a human is. This anthropomorphic lens creates a "backwards-looking" accountability gap. If an AI causes harm, who is responsible? If we view the AI itself as the moral agent, we find ourselves in a vacuum where we cannot meaningfully "punish" or "reform" the model.
Two Competing Visions of Agency
To resolve this accountability gap, the paper utilizes conceptual analysis from political theory to contrast two competing visions of agency:
- Mechanistic Agency: Defined by functionality—the ability to represent the environment, have goals, and take actions. This is the default view in AI (e.g., Reinforcement Learning), where ethics is reduced to an "epistemic problem" (i.e., if the model just had more data, it would be "more" ethical).
- Volitional Agency: Derived from the Aristotelian tradition and the work of philosopher Charles Taylor. This view requires "strong evaluation"—the ability to reflect on the quality of one's own motivations and possess an "original purpose."
Why AI Fails the Volitional Test
When applying these frameworks to current AI practices like Reinforcement Learning from Human Feedback (RLHF) and Constitutional AI, it becomes clear that AI fails the test of volitional agency.
A key distinction lies in original versus derivative purpose. A human has original purpose. A payroll computer—or an LLM—has a derivative purpose; it serves its designer. Because an AI's purpose is always derivative, it can never be a volitional agent.
Under the mechanistic view, the "ideal" AI is essentially a simulator of a perfectly ethical human. However, because such a human doesn't exist, and the model has no "relevant control" over the real-world subjects of its judgments, this "simulator trap" leads directly to a dead-end for accountability (as illustrated in the framework above).
A Paradigm Shift: AI as a Political Outcome
If AI is not a moral agent, what is it? The author argues that "aligning AI to human values" is not a technical optimization problem, but a political one. It is the process of aggregating conflicting human wills.
Therefore, the "ethics" of an AI should be judged by the legitimacy of the process that created it, not the "character" of the model itself. This requires two major shifts in how we evaluate AI:
- From abstract personhood to functional correctness: Instead of asking if an AI is a "good person," we should ask if it is a "good therapist" or a "good tool" for a specific context.
- From model character to process legitimacy: Accountability must be established via the people and processes behind the model, rather than treating the model as an independent moral actor.
Future Directions: Participatory AI
The author acknowledges that viewing AI as a political process is difficult to operationalize. Current methods for aggregating human "wills"—like simple majority voting in RLHF—are often insufficient to capture the complexity of human morality.
To move forward, the paper advocates for "Participatory AI," drawing on Social Choice Theory. Future work must focus on developing better mechanisms for resolving disagreements in AI alignment, expanding the power of those subjected to AI systems to influence their design, and moving beyond simple "preference elicitation" toward active, continuous participation in model governance.
Summary updated