In classical AI planning, an agent uses an "action model" (often written in PDDL) to predict how its actions will change the world. Traditionally, learning these models from data involves making a "best guess" about the preconditions and effects of an action. However, a best guess can be dangerous. If a model incorrectly assumes an action is applicable in a certain state, the resulting plan might lead to catastrophic failure when executed in the real world.
To solve this, Brendan Juba, Roni Stern, and their colleagues have developed a framework for Safe Action Model (SAM) Learning. Instead of trying to find the most likely model, SAM algorithms learn a "conservative" model that prioritizes soundness over completeness.
The Logic of Pessimism
The core intuition behind SAM learning is a "pessimistic" stance toward the unknown. In a standard learning approach, if you haven't seen an action fail in a specific context, you might assume it works. In SAM learning, the logic is reversed: if you haven't explicitly seen an action succeed in a specific context, you assume it might fail.
As described in Efficient, Safe, and Probably Approximately Complete Learning of Action Models, a conservative action model restricts the agent to states where success is certain based on prior observations. This ensures execution safety: any plan generated by the learned model is guaranteed to be valid in the real environment.
Pruning the Precondition Space
How do you build a model that is guaranteed never to be over-optimistic? The mechanism, detailed in Safe Learning of Lifted Action Models by Brendan Juba and Hai Son Le, works through a process of elimination:
- Initialization: Start by assuming that every possible literal (state feature) is a required precondition for an action. This is the most restrictive possible model; the agent assumes the action can never be performed.
- Observation: Observe a successful execution of the action in a specific state.
- Pruning: If a literal was not present in the state where the action succeeded, it cannot be a necessary precondition. Remove it from the set of requirements.
- Intersection: The resulting preconditions are the intersection of all state features present during every observed successful execution.
By starting with "everything is required" and only removing constraints when evidence proves they aren't necessary, the agent ensures it never underestimates the conditions needed for safety.
Soundness vs. Completeness
The primary trade-off in this research direction is between soundness (the model is never wrong when it says an action will work) and completeness (the model knows every possible way an action could work).
As Argaman Mordoch and Enrico Scala argue in their work on Conditional-SAM, in high-stakes domains like autonomous surgery or driving, incompleteness is a minor inconvenience, but unsoundness is a disaster. A conservative agent might say "I don't know how to reach the goal" even if a path exists, but it will never say "I can reach the goal" and then crash midway.
Handling Complexity and Uncertainty
This conservative philosophy has been extended to handle the complexities of real-world environments:
- Partial Observability: In Learning Safe Action Models with Partial Observability, Hai S. Le and Brendan Juba show that even when the agent cannot see the full state of the world, it can maintain safety by only utilizing transitions that have been explicitly verified through partial evidence.
- Stochastic Worlds: For environments with random outcomes, the SAM+ algorithm Learning Probably Approximately Complete and Safe Action Models for Stochastic Worlds uses probability intervals. Instead of a single success probability, it calculates bounds. A plan is only considered "safe" if it succeeds even under the worst-case probabilities within those bounds.
By prioritizing this "safety-first" induction, these researchers have created a path for AI agents to learn and act in the real world without the need for hand-coded, error-free domain models.