Norm Deviation in Multiagent Systems: A Foundation for Responsible Autonomy

Amika M. Singh, Munindar P. SinghIJCAI 2023 · DOI

TL;DR

This paper proposes a novel framework for understanding when and why autonomous agents should deviate from applicable norms, moving beyond simple compliance or conflict resolution. By grounding this framework in Habermas's theory of communicative action and validating it with legal case law, the authors provide a robust conceptual foundation for building responsible AI systems that can navigate complex ethical and social landscapes.

What Problem Does This Paper Address?

As AI agents become increasingly integrated into our lives, ensuring their responsible and prosocial behavior is paramount. A key challenge lies in the inherent tension between an agent's need to respect applicable norms (like laws and policies) and its ability to deviate from them when blind adherence leads to suboptimal or even harmful outcomes. Existing research often focuses on norm conflicts or sanctions for violations, but lacks a clear conceptual framework for justifying norm deviation. This paper addresses this gap by asking: what are the legitimate criteria by which an agent can decide to comply with or deviate from a norm, and how can this legitimacy be established independently of external enforcement?

Methods

The authors propose a conceptual foundation for norm deviation based on Habermas's theory of communicative action. This theory posits that any communication or action in the public sphere can be evaluated based on three validity claims:

  • Objective validity: The factual accuracy of the claim.
  • Subjective validity: The sincerity and truthfulness of the speaker's intentions.
  • Practical validity: The rightness or appropriateness of the claim within a social context.

They adapt these claims to assess the legitimacy of an agent's norm deviation. Crucially, instead of relying solely on theoretical constructs, the paper uses legal case law as an empirical basis for understanding how norms are interpreted and applied in practice. This approach leverages the adversarial nature of litigation to uncover "edge cases" and the reasoning behind deviations, providing a richer understanding than surveys or introspection alone. The authors categorize potential justifications for norm deviation into five areas: promoting primary stakeholder interests, promoting primary stakeholder values, promoting the public interest, protesting a norm, and proximate causation.

Datasets

Not discussed in the paper. The paper uses legal case law as its primary source of empirical validation, rather than traditional datasets.

Key Insights and Ideas

  • Norm deviation is not inherently bad: The paper argues that responsible autonomy requires agents to deviate from norms when necessary, not just comply. Blind compliance can lead to diminished outcomes.
  • Legitimacy from within: The legitimacy of a norm deviation should not rely on external factors like sanctions, but rather on internal justifications that can be evaluated against validity claims.
  • Legal case law as a rich source of insight: The adversarial and context-rich nature of legal disputes provides a more reliable empirical basis for understanding norm deviation than abstract surveys.
  • Habermas's validity claims offer a structured framework: Objective, subjective, and practical validity claims provide a systematic way to evaluate the justifications for an agent's deviation from a norm.
  • Five categories of justified deviation: The paper identifies promoting stakeholder interests/values, public interest, protesting norms, and proximate causation as key areas where norm deviation might be justified.
  • Justified deviation can build trust: Contrary to the idea that only blind compliance builds trust, the paper suggests that justified norm deviations can actually demonstrate an agent's benevolence and integrity.

Key Results

The paper doesn't present quantitative results in the traditional sense. Instead, its key results are conceptual and qualitative:

  • A conceptual framework for justified norm deviation based on Habermas's validity claims.
  • Identification of five categories of reasons for justified norm deviation.
  • Empirical grounding of these categories through analysis of legal precedents from US, Canadian, and English courts.
  • Demonstration of how these legal cases map onto the objective, subjective, and practical validity claims.
  • A proposed argument scheme for reasoning about norm deviation, particularly for client interests.

Limitations and Future Work

  • Western democratic perspective: The authors acknowledge that their underlying theories and case law are primarily from a Western democratic perspective, and cultural effects on norm deviation could be significant. They suggest this could be turned into an advantage by allowing agents to adapt their deviation reasoning based on applicable cultural perspectives.
  • Modeling compositionality: A future challenge is to model norm deviation for organizations versus individual members, analogous to group ability.
  • Capturing accountability: Another open question is how to capture accountability in sociotechnical systems (STS) in light of norm deviation.
  • Computational challenges: While interests are conceptually easy to model (e.g., via utilities), values are more challenging, and protesting norms requires complex historical and social analysis.
  • Domain-specific critical questions: The critical questions for evaluating norm deviation will need to be tailored to specific domains. The authors suggest using human intervention for difficult cases or adapting formal models with oracles trained on data.

Summary updated

BibTeX
@inproceedings{singh2023norm,
  author = {Amika M. Singh and Munindar P. Singh},
  title = {{Norm Deviation in Multiagent Systems: A Foundation for Responsible Autonomy}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/33},
}
Figure