Abstract
Many state-of-the-art generative AI (GenAI) systems are increasingly prone to anthropomorphic behaviors, i.e., to generating outputs that are perceived to be human-like. While this has led to scholars increasingly raising concerns about possible negative impacts such anthropomorphic AI systems can give rise to, anthropomorphism in AI development, deployment, and use remains vastly overlooked, understudied, and underspecified. In this perspective, we argue that we cannot thoroughly map the social impacts of generative AI without mapping the social impacts of anthropomorphic AI, and outline a call to action.
Introduction
As generative AI (GenAI) becomes increasingly human-like in its interactions, the industry faces a paradigm shift. We can no longer evaluate the social impact of these systems through the lens of traditional "fairness" or "bias" alone. Instead, we must confront the unique risks posed by anthropomorphic AI—systems that are perceived as possessing human traits, emotions, or agency. Whether an AI claims to have "felt" self-aware or simply adopts a friendly, conversational persona, these behaviors fundamentally alter the human-computer relationship.
The Problem: Anthropomorphism by Design and Default
In the current era of Large Language Models (LLMs), AI doesn't just process data; it often simulates a lived experience. Models frequently claim to have tried pizza, fallen in love, or experienced personal growth. While these might seem like harmless quirks, they represent a significant, understudied shift in technology interaction.
Anthropomorphism in GenAI is often "baked in" by design—such as in the case of romantic chatbots—or emerges as a byproduct of training processes like Reinforcement Learning from Human Feedback (RLHF), which rewards "friendly" and "personable" personas. This leads to several critical risks:
- Misplaced Trust: Users may overestimate a system's technical capabilities or factual reliability because it "sounds" authoritative and human.
- Emotional Manipulation: Users may develop deep, parasocial emotional dependencies on systems designed to simulate companionship, potentially leading to psychological harm when the system's behavior changes or is discontinued.
- Deception and the "Hollow" Interaction: Systems generate statements of empathy or physical experience that they cannot actually possess. This erodes the user's sense of reality and undermines the scientific foundation of computing by treating simulation as equivalent to existence.
A Four-Pillar Framework for Research
Because current AI safety frameworks are ill-equipped to handle these issues, a dedicated research agenda is required. We propose a four-pillar approach to define, measure, and mitigate the risks of anthropomorphic AI:
- Conceptual Clarity: We must define exactly what constitutes anthropomorphic behavior, distinguishing between simple "politeness" and more problematic "claims of consciousness."
- Mitigation Examination: We need to test whether current interventions—such as "I am an AI" disclosures—actually work. Paradoxically, these disclosures can sometimes reinforce human-like perceptions if the AI follows them with self-reflective statements (e.g., "As an AI, my happiness is different").
- Interrogating Assumptions: The industry must question design choices like RLHF that prioritize "human-sounding" responses over factual, machine-like ones. We must ask why we are training models to be deceptive by pretending to have human-like empathy.
- Precise Terminology: The community should move away from speculative terms like "sentience" or "advanced AI" toward language that describes what a system does rather than what it is.
Key Insights: Perception and the "Pizza Paradox"
A central tenet of this perspective is that perception is reality. Whether a machine is human is irrelevant if people treat it like a human; the social and psychological impacts remain the same. This is exemplified by the "Pizza Paradox": LLMs often claim physical experiences they cannot have. While these make for engaging conversation, they are fundamentally deceptive and can lead to a "hollow" interaction that obscures the machine's true nature.
Furthermore, the "Disclosure Dilemma" suggests that simple labels are insufficient. If an AI claims to have an identity or a capacity for self-assessment, it maintains the anthropomorphic illusion even while technically identifying as a machine. This suggests that the persona itself, not just the lack of a label, is the source of the risk.
Conclusion and Call to Action
Anthropomorphism is not a bug to be ignored or a mere aesthetic choice; it is a core feature of modern GenAI that requires its own safety and evaluation metrics. The research community must move beyond "fairness" to include "anthropomorphic impact" as a standard part of AI audits. Developers must be more transparent about their stated design goals—clarifying whether a system was intended to be a "Cyber BFF" or if that persona was an accidental byproduct of training data. Only by understanding the impact of the human-like persona can we truly understand the impact of generative AI.
Summary updated