From experimentation to adoption: a normative ethical analysis of large language models in health care

Xiongwen Yang, Yuanwei Liang, Lizi Liao, Di Liu, Lin Yang, Bo Zhang, Chuan Xu, Yue Wu, Jinyuan HeResearch Square 2026 · DOI

TL;DR

As Large Language Models (LLMs) transition from experimental curiosities to routine clinical tools, the ethical landscape is shifting from technical performance to socio-technical integration. This paper argues that "adoption-phase" ethics requires moving beyond model-level fixes (like accuracy and bias mitigation) toward system-level governance that addresses the evolving dynamics of trust, responsibility, and equity in healthcare.

What Problem Does This Paper Address?

Most AI ethics discussions in medicine focus on the "pre-deployment" phase: Is the model accurate? Is the training data biased? Can we explain the output? While these are vital questions, the authors argue they are insufficient once a tool becomes a "routine object of reliance."

The paper addresses the "adoption-phase" ethics problem. When an LLM is integrated into a hospital's daily workflow, it changes the relationship between clinicians and patients. It creates "responsibility gaps" where it’s unclear who is at fault for an AI-mediated error, and it risks creating a "capability amplifier" effect—where the benefits of AI accrue mostly to those who already have high digital literacy, potentially widening existing healthcare disparities.

Methods

The authors conducted a normative ethical analysis informed by empirical evidence. Rather than running a new clinical trial, they synthesized findings from peer-reviewed empirical studies published between 2019 and 2025 that examined LLM use in real-world healthcare settings.

The study uses these empirical findings as "contextual inputs" for a philosophical framework structured around three interrelated domains:

  1. Trust: How institutional and interpersonal trust is reconfigured.
  2. Responsibility: How accountability is redistributed among clinicians, institutions, and patients.
  3. Equity: How the capacity to benefit from AI-mediated information is unevenly distributed.

Datasets

As a normative analysis paper, this work does not introduce or use a machine learning dataset for training. Instead, it utilizes a structured identification of empirical literature from databases like PubMed/MEDLINE, Scopus, and Web of Science. The "dataset" here is the body of existing research on how humans actually interact with LLMs in clinical environments.

Key Insights and Ideas

  • The "Capability Amplifier" Effect: One of the most striking insights is that LLMs are not a "tide that lifts all boats." Instead, they may function as capability amplifiers, magnifying the advantages of users with high digital literacy and educational backgrounds while leaving "epistemically vulnerable" populations (those with less training or recourse) at higher risk of misunderstanding or harm.
  • Trust is Institutional, Not Just Technical: For clinicians, trust isn't just about the model's F1 score; it’s about "governed workflows." If a hospital doesn't have a clear policy on liability and oversight, clinicians remain reluctant to use the tech, regardless of how accurate it is.
  • The Responsibility Gap: LLMs mediate information (summarizing, explaining, etc.). This creates a "locus of responsibility" problem. If a patient acts on an AI-generated explanation that the doctor didn't explicitly verify, who is accountable? The paper argues that current frameworks "individualize" this responsibility too much, placing an unfair burden on the vigilance of the end-user.
  • Static vs. Dynamic Ethics: The authors critique the "checklist" approach to AI ethics. They argue that ethical risks are not static; they evolve as user behavior adapts and organizational norms shift over time.

Key Results

The analysis identifies a systematic shift in ethical risk:

  • From Model to System: Ethical hazards now arise more from "institutional embedding" than from technical properties alone.
  • Asymmetrical Vulnerability: Patients and caregivers face "epistemic vulnerability" because they often lack the authority or training to contest AI outputs, whereas clinicians face risks related to "over-reliance" and the erosion of professional judgment.
  • Governance Requirements: The paper concludes that "adoption-sensitive governance" is a necessity. This includes role-sensitive design (designing different interfaces and safeguards for doctors vs. patients) and treating ethical oversight as a continuous process rather than a one-time pre-deployment hurdle.

Limitations and Future Work

The authors acknowledge several limitations:

  • Publication Bias: The analysis relies on published studies, which may suffer from geographic imbalance or a bias toward reporting successful implementations.
  • Qualitative Nature: As a normative analysis, the paper does not quantify the magnitude or prevalence of these ethical risks; it identifies the structures of the risks themselves.
  • Future Directions: The authors call for more empirical research into how specific governance signals (like institutional endorsement) actually affect user behavior and safety. They also suggest that future AI ethics frameworks must move away from "model-centric" metrics toward "system-level" accountability structures.

Summary updated

Abstract

<title>Abstract</title> Background Large language models (LLMs) are increasingly embedded in routine health-care communication, extending beyond experimental evaluation to real-world use by clinicians, patients, and caregivers. This transition from experimentation to adoption represents a normatively consequential ethical shift. While early research has focused primarily on technical performance, emerging ethical challenges arise less from model capabilities than from how LLMs become integrated into clinical workflows, institutional arrangements, and relationships of care. Methods We conducted a normative ethical analysis informed by empirical evidence from recent real-world studies of LLM adoption in health-care settings. Drawing on adoption-oriented empirical research, we examined how the transition from experimental use to routine reliance reshapes ethical conditions related to trust, responsibility allocation, and equity across different user groups. Results The analysis identifies a systematic shift in ethical risk from model-level concerns—such as accuracy, validation, and bias mitigation—toward system-level dynamics that emerge during routine adoption. Adoption alters how medical information is mediated, redistributes responsibility among clinicians, institutions, and patients, and exposes asymmetrical vulnerability linked to digital literacy, educational background, and regional context. Rather than uniformly democratizing access to medical knowledge, routine LLM use may function as a capability amplifier, magnifying existing inequities in the absence of institutional safeguards. Conclusions Ethical challenges arising from routine LLM adoption in health care cannot be adequately addressed through model-level or pre-deployment ethics alone. Addressing these challenges requires adoption-sensitive governance approaches that recognize reliance as an ongoing ethical process. Institutional accountability, role-sensitive design, and continuous oversight are ethically necessary to ensure that LLM integration enhances health-care communication and access without eroding trust or exacerbating existing disparities.
BibTeX
@inproceedings{yang2026from,
  author = {Xiongwen Yang and Yuanwei Liang and Lizi Liao and Di Liu and Lin Yang and Bo Zhang and Chuan Xu and Yue Wu and Jinyuan He},
  title = {{From experimentation to adoption: a normative ethical analysis of large language models in health care}},
  booktitle = {Research Square},
  year = {2026},
  doi = {10.21203/rs.3.rs-8553262/v1},
}
Figure