TL;DR
This paper proposes the creation of a professional association for Artificial Intelligence (AI) companies to establish and enforce a code of practice. This mechanism aims to raise the standards of AI products, build public trust, and protect both the AI field and society from the potential negative impacts of substandard or misleading AI offerings.
What Problem Does This Paper Address?
The authors identify a critical challenge facing the burgeoning AI industry: the lack of a robust mechanism to ensure the quality and reliability of AI products. Unlike established scientific fields with peer review processes, the commercial AI landscape is vulnerable to "charlatans and incompetents" who could exploit the current enthusiasm for AI by selling overrated or useless products. Such failures could lead to a public backlash, damaging the reputation of AI and hindering its future development and adoption. The paper argues that this is not just an internal industry problem but also a societal one, as AI products are poised to play a significant role, necessitating protection from exploitation and harm.
Methods
The core proposal is the establishment of a professional association for AI vendors. This association would operate on a self-regulatory model, relying on a code of practice and a panel to hear complaints. Key aspects of the proposed method include:
- Membership-based regulation: Companies would voluntarily join the association, with membership contingent on adhering to a code of practice and maintaining high standards.
- Code of Practice: A set of guidelines outlining what constitutes reasonable behavior for AI vendors, focusing on accurate descriptions of product abilities and limitations.
- Complaint Panel: A body to investigate and adjudicate complaints from customers or other interested parties.
- Passive Role: The association would primarily react to complaints rather than actively pre-vetting products, due to practical considerations like cost and the difficulty of setting definitive standards for a rapidly evolving field.
- Case Law Development: The panel's decisions would build a body of "case law" to guide future judgments and establish what constitutes reasonable practice over time.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- The "Grey Area" of AI Claims: The paper highlights the difficulty in defining what constitutes an "exaggerated" or "misleading" claim in AI. Terms like "understand" or "programming" are ambiguous, making it hard to draw a clear line between genuine limitations and deceptive marketing.
- The Importance of Describing Limitations: Beyond avoiding false claims, vendors must actively describe the limitations of their AI products. This is crucial for managing customer expectations and preventing disappointment.
- Balancing Transparency and Trade Secrets: A significant challenge is how to accurately describe product limitations without revealing proprietary information about the underlying techniques or software, especially given the weaker legal protections for software compared to patents.
- Passive Regulation as a Practical Necessity: The authors argue for a passive, complaint-driven regulatory approach as more feasible for a nascent industry. This shifts the burden of proof to the complainant but relies on the expertise of AI customers (often researchers themselves) to identify and report issues.
- Societal Responsibility: The proposal extends beyond mere industry self-interest, emphasizing the ethical imperative to protect society from potential harms and exploitation by AI companies.
Key Results
This paper is a proposal for a mechanism, not an empirical study with quantitative results. The "results" are the conceptual framework and the proposed code of practice. The key outcome is the detailed outline of a self-regulatory association and its operational principles.
Limitations and Future Work
- Burden on the Customer: The passive regulatory approach places the onus on the customer to identify and report issues, which might be a barrier for less technically savvy users.
- Risk of Becoming a Trade Protection Society: The authors acknowledge the danger that the association could devolve into a mechanism for protecting existing members from competition rather than upholding standards.
- Legal Challenges: The association might face legal action (e.g., for libel) from disgruntled vendors, necessitating a legal defense fund.
- International Scope: The proposal acknowledges the international nature of the AI market and suggests the association should also be international, though the practicalities are complex.
- Future Work: The authors explicitly state the paper is a "vehicle for discussion" and welcome feedback, particularly on the proposed code of practice. They suggest that as the AI field matures and companies become wealthier, the association could evolve towards a more active role. They also note the need to consider how to address AI products that incorporate non-AI techniques and the gap in regulation for government organizations.
Summary updated