AI Ethics - Too Principled to Fail?

Brent D. MittelstadtarXiv 2019 · arXiv · OpenReview

Abstract

AI Ethics is now a global topic of discussion in academic and policy circles. At least 84 public-private initiatives have produced statements describing high-level principles, values, and other tenets to guide the ethical development, deployment, and governance of AI. According to recent meta-analyses, AI Ethics has seemingly converged on a set of principles that closely resemble the four classic principles of medical ethics. Despite the initial credibility granted to a principled approach to AI Ethics by the connection to principles in medical ethics, there are reasons to be concerned about its future impact on AI development and governance. Significant differences exist between medicine and AI development that suggest a principled approach in the latter may not enjoy success comparable to the former. Compared to medicine, AI development lacks (1) common aims and fiduciary duties, (2) professional history and norms, (3) proven methods to translate principles into practice, and (4) robust legal and professional accountability mechanisms. These differences suggest we should not yet celebrate consensus around high-level principles that hide deep political and normative disagreement.

Introduction

In recent years, the tech industry and policy-making bodies have seen an explosion of "AI Ethics" guidelines. With over 80 initiatives publishing high-level principles, a clear trend has emerged: a convergence on a set of core values—respect for autonomy, prevention of harm, fairness, and explicability. This "principlist" approach is intentionally modeled after the four pillars of medical ethics, which have successfully guided clinical practice for decades.

However, this paper argues that the tech industry’s reliance on high-level principles is a flawed strategy. While these principles provide a veneer of ethical rigor, they often serve as a form of "virtue signaling" or "ethics washing" designed to delay meaningful government regulation. By comparing the structural foundations of medicine and AI development, we can see why a model that works for doctors cannot be seamlessly applied to software engineers and data scientists.

The Structural Gap: Why AI is Not Medicine

The success of medical ethics does not stem from the principles themselves, but from the institutional and professional infrastructure that supports them. AI development currently lacks four critical components that make medical principlism effective.

1. Lack of Fiduciary Duties

In medicine, the relationship between a doctor and a patient is governed by a fiduciary duty—a legal and moral obligation to prioritize the patient's well-being above all else. In contrast, AI developers are primarily accountable to corporate shareholders. This creates a fundamental misalignment of incentives. When ethical considerations conflict with profit margins or market share, the lack of a legal fiduciary requirement means that ethics are often sidelined in favor of commercial interests.

2. Absence of Professional History and Norms

Medical ethics was not created in a vacuum; it was forged through centuries of practice and refined in response to historical crises and atrocities, such as those that led to the Nuremberg Code. This history has created a unified professional identity and a shared moral language. AI development, by comparison, is a young and fragmented field. It lacks a cohesive professional identity or a historical tradition of ethical deliberation, making it difficult to ground high-level principles in a shared sense of professional responsibility.

3. The Translation Problem

High-level concepts like "fairness," "justice," or "dignity" are what philosophers call "essentially contested concepts." Reasonable people—and different cultures—often disagree on what they mean in practice. Medicine handles this through established institutional mechanisms, such as clinical ethics boards, which translate abstract principles into specific, context-aware actions. In the AI industry, these terms are often left to the interpretation of individual developers or marketing departments, leading to a form of moral relativism where "ethics" becomes whatever a company decides it is.

4. Toothless Accountability

Perhaps the most significant difference is the lack of enforcement. Medicine is a licensed profession with strict entry requirements, malpractice laws, and the power to revoke a practitioner's ability to work. AI development has no such mechanisms. Current ethical codes in AI are almost entirely self-regulatory. Research has shown that without punitive consequences or professional licensing, these codes are largely ineffective at changing behavior in high-stakes environments.

Beyond Principles: A Path Forward

If high-level principles are insufficient, how should the field of AI ethics evolve? The paper suggests shifting the focus from vague statements to concrete structural changes:

  • Licensing for High-Risk AI: Rather than attempting to license all software engineering, formal licensing and certification should be required for those developing high-stakes systems, such as facial recognition for policing or algorithms used in criminal sentencing.
  • Organizational Accountability: Ethics should not be framed as the responsibility of individual "bad apple" developers. Instead, the focus must shift to the collective failure of unethical business models and organizational structures.
  • Bottom-Up Empirical Research: We must move away from top-down, generalist principles and toward case-specific research. By studying how ethics are negotiated on the ground within private-sector environments, we can develop more practical, context-specific guidelines.
  • Democratic Intervention: Ultimately, ethics cannot be a substitute for law. To protect the public, we must move beyond voluntary codes toward binding requirements for transparent ethical reviews, independent auditing, and inclusive design.

Conclusion

The current consensus on AI ethics principles is a "false dawn." While these principles cost companies nothing to sign, they offer the public a false sense of security while shielding the industry from necessary regulation. To achieve truly ethical AI, we must stop treating ethics as a technical "patch" and start building the professional, legal, and democratic institutions necessary to hold the technology—and its creators—accountable.

Summary updated

Abstract

AI Ethics is now a global topic of discussion in academic and policy circles. At least 84 public-private initiatives have produced statements describing high-level principles, values, and other tenets to guide the ethical development, deployment, and governance of AI. According to recent meta-analyses, AI Ethics has seemingly converged on a set of principles that closely resemble the four classic principles of medical ethics. Despite the initial credibility granted to a principled approach to AI Ethics by the connection to principles in medical ethics, there are reasons to be concerned about its future impact on AI development and governance. Significant differences exist between medicine and AI development that suggest a principled approach in the latter may not enjoy success comparable to the former. Compared to medicine, AI development lacks (1) common aims and fiduciary duties, (2) professional history and norms, (3) proven methods to translate principles into practice, and (4) robust legal and professional accountability mechanisms. These differences suggest we should not yet celebrate consensus around high-level principles that hide deep political and normative disagreement.
BibTeX
@misc{mittelstadt2019ai,
  author = {Brent D. Mittelstadt},
  title = {{AI Ethics - Too Principled to Fail?}},
  year = {2019},
  eprint = {1906.06668},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.06668},
}
Figure