The Jiminy Advisor: Moral Agreements Among Stakeholders Based on Norms and Argumentation

Beishui Liao, Pere Pardo, Marija Slavkovik, Leendert van der Torre

Abstract

Autonomous systems are increasingly integrated into society, operating under the influence of manufacturers, legal frameworks, and end-users. Each of these stakeholders holds distinct ethical views that can conflict. We propose Jiminy, an ethical recommendation component that uses normative systems and formal argumentation to reach moral agreements among stakeholders. Jiminy models stakeholder views as normative systems and employs a multi-level resolution strategy—adding arguments, adding attacks, and revising priorities—to resolve dilemmas. This framework not only facilitates collaborative decision-making but also generates structured explanations for its ethical recommendations.

Introduction: The "Morality by Committee" Problem

As autonomous systems like smart home assistants and self-driving cars become ubiquitous, they face complex moral choices where the interests of different stakeholders clash. A manufacturer may prioritize liability limitation, a user may demand absolute privacy, and a government requires adherence to the law. For instance, should a smart speaker report a detected domestic threat (legal requirement) or protect the user's privacy (user preference)?

Existing approaches often rely on "black box" AI or hardcoded single-theory ethics, which lack transparency and flexibility. Jiminy addresses this challenge of multi-stakeholder ethical integration by treating diverse moral inputs as competing arguments within a structured, symbolic AI framework.

The Jiminy Architecture

Jiminy is built on two pillars: Normative Systems and Formal Argumentation. Each stakeholder's ethical perspective is modeled as a set of rules, including constitutive norms (defining situations) and regulative norms (defining obligations).

The Jiminy architecture for ethical recommendation and explanation.

As shown in the architecture above, the system processes sensory input and knowledge to identify "morally sensitive situations." It then utilizes an argumentation reasoning engine to evaluate possible options against the normative systems provided by stakeholders. The output is a moral recommendation accompanied by a generated explanation.

Multi-Level Conflict Resolution

To reconcile conflicting norms, Jiminy employs a four-level resolution strategy:

  1. Individual Level: Checks for internal dilemmas within a single stakeholder's view.
  2. Combined Level: Merges arguments into a single graph to see if standard defeat logic resolves the conflict.
  3. Integrated Level: Merges the underlying rules, allowing the system to reason across stakeholders (e.g., a manufacturer's factual data triggering a legal obligation).
  4. Reduced Level: Applies context-sensitive "meta-norms" to decide which stakeholder takes precedence (e.g., Law overrides User in life-threatening scenarios).
An abstract representation of an argumentation framework where nodes represent moral claims and arrows represent defeat relations.

In this framework, ethical views are transformed into a directed graph of arguments. If two arguments conflict, a "defeat" relation is established based on a hierarchy of priorities. This allows the system to reach a logically sound conclusion even when initial inputs are contradictory.

Key Insights and Results

  • Argumentation as Explanation: Because the system uses a graph of arguments and defeats, the explanation for a decision is simply a trace of the winning argument path. It can explicitly state: "Action A was chosen because the Legal Code's safety requirement defeated the User's privacy preference in this context."
  • Computational Efficiency: By using propositional rules and argumentation rather than complex modal logics, the system remains efficient and auditable.
  • Formal Soundness: The framework satisfies rationality postulates, ensuring that recommendations are logically consistent and that sub-arguments do not contradict their parent arguments.

Conclusion and Future Work

Jiminy demonstrates how symbolic AI can resolve the "morality by committee" problem in a transparent, explainable way. While currently reliant on pre-structured norms, future work aims to address the knowledge engineering bottleneck by incorporating machine learning to help stakeholders refine their moral theories over time. Additionally, expanding the framework to handle the complexities of international law remains a significant area for exploration.

Summary updated

Abstract

An autonomous system is constructed by a manufacturer, operates in a society subject to norms and laws, and interacts with end users. All of these actors are stakeholders affected by the behavior of the autonomous system. We address the challenge of how the ethical views of such stakeholders can be integrated in the behavior of an autonomous system. We propose an ethical recommendation component called Jiminy which uses techniques from normative systems and formal argumentation to reach moral agreements among stakeholders. A Jiminy represents the ethical views of each stakeholder by using normative systems, and has three ways of resolving moral dilemmas that involve the opinions of the stakeholders. First, the Jiminy considers how the arguments of the stakeholders relate to one another, which may already resolve the dilemma. Secondly, the Jiminy combines the normative systems of the stakeholders such that the combined expertise of the stakeholders may resolve the dilemma. Thirdly, and only if these two other methods have failed, the Jiminy uses context-sensitive rules to decide which of the stakeholders take preference over the others. At the abstract level, these three methods are characterized by adding arguments, adding attacks between arguments, and revising attacks between arguments. We show how a Jiminy can be used not only for ethical reasoning and collaborative decision-making, but also to provide explanations about ethical behavior.
BibTeX
@article{liao2023jiminy,
  author = {Beishui Liao and Pere Pardo and Marija Slavkovik and Leendert van der Torre},
  title = {{The Jiminy Advisor: Moral Agreements Among Stakeholders Based on Norms and Argumentation}},
  journal = {J. Artif. Intell. Res.},
  year = {2023},
  doi = {10.1613/jair.1.14368},
}

Loading related content...

Figure