The First Law of Robotics (A Call to Arms)

Daniel S. Weld, Oren EtzioniSafety and Security in Multiagent Systems 1994 · DOI

Introduction

Isaac Asimov’s "Three Laws of Robotics" have long served as a philosophical cornerstone for the ethics of artificial intelligence. However, for the engineers tasked with building autonomous agents—whether physical robots or software-based "softbots"—these laws remain frustratingly informal. As we deploy agents into complex environments like file systems or physical workspaces, we encounter a fundamental challenge: how do we ensure an agent does not cause "harm" while pursuing a user-defined goal?

A classic example of this problem occurs in file management. If an agent is instructed to "reduce disk usage," it might technically succeed by deleting the user's irreplaceable research papers. While the literal command is satisfied, the outcome is catastrophic. This paper proposes a formal, computationally tractable framework for implementing Asimov’s First Law ("A robot may not injure a human being...") within the context of automated planning.

A Formal Framework for Safety and Tidiness

Rather than relying on complex utility functions that turn every task into an intractable optimization problem, the authors propose a "satisficing" approach. They introduce two logical primitives based on Action Description Language (ADL) to govern agent behavior:

  1. dont-disturb(C) (Safety): This constraint mandates that an agent must not create any new violations of a condition $C$. Crucially, this is an incremental requirement. If a condition $C$ is already violated in the initial state, the agent is not obligated to fix it, but it is strictly forbidden from making the situation worse. In this framework, safety constraints always override the agent's primary goals.
  2. restore(C) (Tidiness): This primitive ensures that an agent "cleans up" after itself. For instance, if an agent must decompress a file to read it, it should re-compress it afterward. Unlike safety, tidiness is secondary to the primary goal; if a goal cannot be achieved without leaving a minor mess, the agent prioritizes the goal over the restoration.

The Planning Algorithm

The authors integrate these primitives into a planning synthesis model. They define a violation function that checks proposed actions against safety and tidiness constraints. When a potential harm is detected during the planning phase, the agent can employ three strategies:

  • Confront: Add a subgoal to ensure that a conditional harmful effect is never triggered.
  • Evade: Use goal regression to ensure the harmful condition remains false throughout the execution of the action.
  • Refuse: Backtrack and seek an entirely different path to the goal that does not involve the harmful action.

This hierarchy establishes a clear chain of command: Safety > User Goals > Tidiness.

Computational Tractability and Insights

A significant contribution of this work is the proof of its efficiency. The authors demonstrate that detecting potential harm (the violation test) can be performed in $O(ec)$ time, where $e$ represents the number of action effects and $c$ represents the complexity of the constraint. This polynomial-time complexity makes the framework suitable for real-time planning in autonomous systems.

Furthermore, the authors observe that many domains—such as UNIX file systems or household environments—exhibit "trivial serializability." Because most actions in these domains are reversible, the agent can often handle "tidiness" in a simple second phase of planning. The agent first finds a safe path to the goal and then appends the necessary cleanup actions, avoiding the need to re-plan the entire mission from scratch.

Limitations and Future Directions

While this framework provides a robust starting point for engineering safe agents, several challenges remain:

  • The Closed World Assumption: The model assumes the agent possesses complete and accurate information. In the real world, harm often results from ignorance rather than malice or poor planning.
  • Thriftiness: The current logic does not effectively handle resource management (e.g., preventing an agent from spending excessive funds to achieve a minor goal).
  • The Problem of Inaction: The second half of Asimov’s First Law—preventing harm through inaction—is significantly more difficult to formalize. This "vigilance" requires the agent to understand human goals and intervene when those goals are threatened by external forces, a task that remains an open challenge in AI research.
  • Dynamic Environments: The current model assumes a static world where the agent is the sole cause of change, ignoring the complexities introduced by other agents or human actors.

By translating literary concepts into formal logical definitions, this work moves us closer to agents that can be trusted to operate autonomously in human-centric environments.

Summary updated

BibTeX
@inproceedings{weld1994first,
  author = {Daniel S. Weld and Oren Etzioni},
  title = {{The First Law of Robotics (A Call to Arms)}},
  booktitle = {Safety and Security in Multiagent Systems},
  year = {1994},
  doi = {10.1007/978-3-642-04879-1\_7},
}
Figure