Moral Permissibility of Action Plans

Felix Lindner, Robert Mattmüller, Bernhard NebelAAAI 2019 · DOI

Introduction

As autonomous systems—ranging from self-driving vehicles to household robotics—become increasingly integrated into daily life, the necessity for ethical decision-making frameworks becomes paramount. Traditional machine ethics has largely focused on evaluating isolated actions in vacuum. However, real-world agents operate through automated planning, executing sequences of actions to achieve specific goals. A single action might appear benign or harmful in isolation, but its moral weight often depends on its role within a broader trajectory.

This research, originally presented at AAAI-19, addresses this gap by providing a formal framework for evaluating the moral permissibility of automated action plans. By extending classical planning formalisms with moral valuations and counterfactual reasoning, the authors categorize the computational complexity of verifying plans under diverse ethical theories, from simple Deontology to the nuanced Principle of Double Effect.

Formalizing Plan Ethics

To move from "action ethics" to "plan ethics," the authors build upon the SAS+ planning formalism. They extend this standard model with three critical components:

  1. Conditional Effects and Exogenous Events: This allows the model to account for environmental changes that occur independently of the agent's actions, providing a more realistic simulation of the world.
  2. Moral Valuations: The framework introduces a utility function $u$ that assigns real-numbered values to both actions (e.g., the act of "pushing" might have a negative valuation) and states (e.g., "safety" having a high positive valuation).
  3. Counterfactual Semantics: A key innovation of this paper is the use of counterfactuals to distinguish between intended means and unintended side effects. By asking, "What would have happened if a specific effect were omitted?", the framework can mathematically determine if a harm was a necessary step toward a goal.

Ethical Theories in Planning

The framework formalizes several prominent ethical principles into computable logic:

  • Deontology: Focuses on the inherent rightness of actions. A plan is permissible if no individual action within the sequence is forbidden.
  • Utilitarianism: Focuses on outcome maximization. A plan is permissible only if it results in the highest possible total utility compared to all other possible plans.
  • Do-No-Harm (Standard & Asimovian): These principles focus on avoiding actions that cause harm which would not have occurred otherwise.
  • Principle of Double Effect (PDE): This sophisticated doctrine permits an action that causes a "bad" side effect only if the action itself is not inherently wrong, the bad effect is not the means to the goal, and the positive outcome outweighs the negative.

Validation via Moral Dilemmas

The authors validated their framework using classic ethical benchmarks, specifically the Trolley Problem and the Footbridge Problem. In the standard Trolley Problem (pulling a lever to save five by killing one), the framework identifies the death as a side effect. In the Footbridge Problem (pushing a man to stop the trolley), the death is identified as a means to the goal. The framework correctly aligns with common human intuition: the Principle of Double Effect permits the former but forbids the latter.

The Complexity Hierarchy of Ethics

A significant contribution of this work is the classification of the computational effort required to verify different ethical theories. The authors establish a "Complexity Hierarchy":

  • Deontic Validation (Polynomial - $P$): Verifying a list of actions against a set of forbidden acts is computationally simple.
  • Do-No-Harm / Double Effect (co-NP-complete): These require checking subsets of actions and effects to determine if harm was avoidable or served as a necessary means, increasing the difficulty.
  • Utilitarian / Asimovian Validation (PSPACE-complete): These are the most computationally demanding. To verify a utilitarian plan, the system must prove that no other possible plan in the entire state space could produce a better outcome.

Discussion and Future Directions

While this framework provides a rigorous mathematical foundation for ethical planning, several challenges remain. The "modeling burden" is significant; the system assumes that a human designer has already accurately quantified utility values for all possible states and actions. Furthermore, the paper focuses on plan validation (checking if a given plan is okay) rather than plan synthesis (generating an ethical plan from scratch).

As AI systems move into high-stakes environments, the PSPACE-completeness of utilitarian checks suggests that real-time ethical verification will require significant engineering breakthroughs. Future work must bridge the gap between these formal logical proofs and the real-time, ambiguous nature of human environments.

Summary updated

BibTeX
@inproceedings{lindner2019moral,
  author = {Felix Lindner and Robert Mattmüller and Bernhard Nebel},
  title = {{Moral Permissibility of Action Plans}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33017635},
}

Loading related content...

Figure