Consent as a Foundation for Responsible Autonomy

Munindar P. SinghAAAI 2022 · DOI

Abstract

Responsible AI is often treated as a static design-time checklist or a set of statistical fairness metrics. This paper proposes consent as a fundamental building block for "responsible autonomy," enabling intelligent agents to dynamically navigate complex social and ethical boundaries within sociotechnical systems. By moving beyond simple "Notice and Choice" models, the framework leverages Habermas’s theory of communicative action to define valid consent across objective, subjective, and practical dimensions.

Introduction

Current approaches to AI ethics frequently rely on "static" methodologies, such as developer checklists or statistical properties like ensuring an algorithm isn't biased against a specific demographic. While valuable, these methods fail to guide autonomous agents operating in real-world environments where contexts shift and decisions impact individuals in real-time. Furthermore, mainstream AI ethics often focuses on artificial "moral quandaries" like the Trolley Problem, which lack the nuance of ongoing human interactions.

The prevailing digital consent standard—"Notice and Choice"—is fundamentally broken, often ignoring power imbalances and the complexity of data usage. This paper seeks to establish a foundation for agents to become truly responsible by understanding and respecting human autonomy through a robust model of consent.

Methodology

This conceptual work draws from philosophy, law, and multiagent systems (MAS) to construct a framework for consent. The core methodology integrates:

  1. Sociotechnical Systems (STS) Modeling: AI is viewed as part of a system of "principals" (humans/organizations) and "agents" (software) governed by social and legal norms.
  2. Habermas’s Theory of Communicative Action: The framework adapts Jürgen Habermas’s validity claims to define what constitutes "valid" consent. This shifts consent from a binary toggle to a multi-dimensional interaction involving:
  3. Objective claims: Is the consent observable and uncoerced?
  4. Subjective claims: Does the consenter possess the capacity, intention, and correct beliefs?
  5. Practical claims: Is the consent aligned with contextual laws and norms?

Consent acts as "moral magic," transforming the ethical status of an action (e.g., the difference between borrowing and stealing). To operationalize this for AI, the paper provides a rigorous breakdown of consent criteria.

Table 1: Criteria for valid consent based on Habermas’s validity claims.

As shown in Table 1, these criteria span visibility, free will, truth, mental capacity, and power dynamics. This taxonomy provides a roadmap for how an autonomous agent might reason about its interactions to ensure they remain ethically grounded.

Research Roadmap

The paper outlines several key areas for future research to realize consent-based autonomy:

  • Social Semantics: Developing formal languages for agents to negotiate consent.
  • Consent-Based Design: Treating consent as a "first-class" requirement in software engineering, similar to goals or dependencies.
  • Verification: Creating formal methods to certify that sociotechnical systems respect consent.
  • Interaction Models: Improving human-agent dialogues to ensure consent is explicit without becoming tedious or causing "consent fatigue."

Discussion and Limitations

While consent offers a powerful foundation, it is difficult to model computationally. Key challenges include the "tedium problem," where frequent requests for consent degrade usability. Additionally, translating philosophical and legal nuances into computational logic remains a significant hurdle. Finally, building agents capable of recognizing and mitigating "subjugation" or power imbalances in digital environments is a critical technical and social challenge.

Summary updated

BibTeX
@inproceedings{singh2022consent,
  author = {Munindar P. Singh},
  title = {{Consent as a Foundation for Responsible Autonomy}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21494},
}
Figure