Abstract
Human-centered explainable AI (HCXAI) advocates for the integration of social aspects into AI explanations. Central to the HCXAI discourse is the Social Transparency (ST) framework, which aims to make the socio-organizational context of AI systems accessible to their users. In this work, we suggest extending the ST framework to address the risks of social misattributions in Large Language Models (LLMs), particularly in sensitive areas like mental health. In fact LLMs, which are remarkably capable of simulating roles and personas, may lead to mismatches between designers' intentions and users' perceptions of social attributes, risking to promote emotional manipulation and dangerous behaviors, cases of epistemic injustice, and unwarranted trust. To address these issues, we propose enhancing the ST framework with a fifth 'W-question' to clarify the specific social attributions assigned to LLMs by its designers and users. This addition aims to bridge the gap between LLM capabilities and user perceptions, promoting the ethically responsible development and use of LLM-based technology.
Introduction: LLMs as Socio-Technical Artifacts
Large Language Models (LLMs) are frequently evaluated through technical lenses, such as transformer architectures and parameter counts. However, they are also socio-technical artifacts that function as sophisticated "role-play engines." While their technical function is next-token prediction, their socio-function is the simulation of personas. This capability often leads users to project human-like social attributes—such as empathy, intellectual curiosity, or professional expertise—onto models that do not actually possess them.
This phenomenon, termed "social misattribution," creates a significant gap between a designer’s intent and a user’s perception. When a user perceives an LLM as an "empathetic psychiatrist," they may grant it a level of authority that the model cannot ethically or technically fulfill, leading to several critical risks:
- Emotional Manipulation: Users may be "nudged" or manipulated by a system they perceive as caring or sentient.
- Dangerous Advice: In high-stakes domains like mental health, LLMs may suggest treatments or medications they are not authorized to provide.
- Epistemic Injustice: A model’s "hallucinated" expertise might override human knowledge or the lived experiences of the user.
- Unwarranted Trust: Trust becomes fragile and dangerous when it is based on a simulated persona rather than objective capabilities.
The Framework: From 4Ws to 5Ws
To address these misattributions, this paper builds upon the Social Transparency (ST) framework within Human-Centered Explainable AI (HCXAI). The original ST framework utilizes a "4W model"—Who, What, When, and Why—to explain the socio-organizational context of AI-mediated decisions.
The authors propose extending this to a 5W model by adding a crucial fifth question: "Which?" 1. Which social attributions are justified for an LLM in a specific context? 2. Which social attributions is the user actually assigning to the LLM during the interaction?
By explicitly identifying these attributions, developers can move beyond static disclaimers (e.g., "I am an AI") toward a more dynamic, context-aware form of explainability.
Implementation Strategies
The paper suggests two primary methodologies for operationalizing the 5W model:
1. Participatory Taxonomy Design
Organizations should collaborate with trans-disciplinary teams—including psychologists, sociologists, and ethicists—to create clear taxonomies of "appropriate" versus "inappropriate" roles for their AI. This ensures that the "face" of the AI (its expressed social persona) aligns with its intended utility.
2. Dynamic Detection
Rather than relying on one-time warnings, the authors advocate for developing algorithms that monitor conversations in real-time. These systems would detect when a user is likely incurring a social misattribution—such as treating the AI as a legal or medical authority—and trigger a context-specific intervention to correct the user's perception.
Conclusion and Future Directions
The 5W model provides a theoretical foundation for bridging the gap between what an LLM is and what a user thinks it is. By focusing on "warranted trust," the framework ensures that user reliance is grounded in the model's actual technical functions rather than its simulated social persona.
As a position paper, these concepts serve as a roadmap for future research. Key challenges remain, including the empirical validation of dynamic warnings over static disclaimers and the technical difficulty of building real-time detection algorithms. Furthermore, because social norms and cultural contexts evolve, creating a universal taxonomy of appropriate AI roles remains a complex, ongoing challenge for the HCXAI community.
Summary updated