Abstract
Autonomous systems are increasingly integrated into society, operating under the influence of manufacturers, legal frameworks, and end-users. Each of these stakeholders holds distinct ethical views that can conflict. We propose Jiminy, an ethical recommendation component that uses normative systems and formal argumentation to reach moral agreements among stakeholders. Jiminy models stakeholder views as normative systems and employs a multi-level resolution strategy—adding arguments, adding attacks, and revising priorities—to resolve dilemmas. This framework not only facilitates collaborative decision-making but also generates structured explanations for its ethical recommendations.
Introduction: The "Morality by Committee" Problem
As autonomous systems like smart home assistants and self-driving cars become ubiquitous, they face complex moral choices where the interests of different stakeholders clash. A manufacturer may prioritize liability limitation, a user may demand absolute privacy, and a government requires adherence to the law. For instance, should a smart speaker report a detected domestic threat (legal requirement) or protect the user's privacy (user preference)?
Existing approaches often rely on "black box" AI or hardcoded single-theory ethics, which lack transparency and flexibility. Jiminy addresses this challenge of multi-stakeholder ethical integration by treating diverse moral inputs as competing arguments within a structured, symbolic AI framework.
The Jiminy Architecture
Jiminy is built on two pillars: Normative Systems and Formal Argumentation. Each stakeholder's ethical perspective is modeled as a set of rules, including constitutive norms (defining situations) and regulative norms (defining obligations).
As shown in the architecture above, the system processes sensory input and knowledge to identify "morally sensitive situations." It then utilizes an argumentation reasoning engine to evaluate possible options against the normative systems provided by stakeholders. The output is a moral recommendation accompanied by a generated explanation.
Multi-Level Conflict Resolution
To reconcile conflicting norms, Jiminy employs a four-level resolution strategy:
- Individual Level: Checks for internal dilemmas within a single stakeholder's view.
- Combined Level: Merges arguments into a single graph to see if standard defeat logic resolves the conflict.
- Integrated Level: Merges the underlying rules, allowing the system to reason across stakeholders (e.g., a manufacturer's factual data triggering a legal obligation).
- Reduced Level: Applies context-sensitive "meta-norms" to decide which stakeholder takes precedence (e.g., Law overrides User in life-threatening scenarios).
In this framework, ethical views are transformed into a directed graph of arguments. If two arguments conflict, a "defeat" relation is established based on a hierarchy of priorities. This allows the system to reach a logically sound conclusion even when initial inputs are contradictory.
Key Insights and Results
- Argumentation as Explanation: Because the system uses a graph of arguments and defeats, the explanation for a decision is simply a trace of the winning argument path. It can explicitly state: "Action A was chosen because the Legal Code's safety requirement defeated the User's privacy preference in this context."
- Computational Efficiency: By using propositional rules and argumentation rather than complex modal logics, the system remains efficient and auditable.
- Formal Soundness: The framework satisfies rationality postulates, ensuring that recommendations are logically consistent and that sub-arguments do not contradict their parent arguments.
Conclusion and Future Work
Jiminy demonstrates how symbolic AI can resolve the "morality by committee" problem in a transparent, explainable way. While currently reliant on pre-structured norms, future work aims to address the knowledge engineering bottleneck by incorporating machine learning to help stakeholders refine their moral theories over time. Additionally, expanding the framework to handle the complexities of international law remains a significant area for exploration.
Summary updated