Abstract
POIROT (Plan Order Induction by Reasoning from One Trial) is an integrated learning framework designed to master complex, hierarchical web service procedures from a single human demonstration. By "stitching" together hypotheses from specialized inductive, deductive, and abductive learning algorithms into a unified executable model, POIROT achieves performance comparable to human subjects in complex domains like medical evacuation planning.
Introduction
Automating complex web service tasks, such as multi-leg travel booking or medical evacuation coordination, typically requires extensive manual programming or massive datasets. In contrast, humans can often grasp the "gist" of a procedure from a single demonstration. POIROT addresses this gap by employing a "team" of learners that collaborate to build a complete model from a "semantic trace"—a log of service calls annotated with their functional meanings.
The core challenge is that no single machine learning algorithm serves as a universal solution. Inductive methods excel at sequence patterns, deductive methods at causal links, and abductive methods at explaining steps based on background knowledge. POIROT integrates these diverse strategies to overcome the limitations of individual learners.
Methods and Architecture
POIROT’s architecture centers on a hypothesis blackboard and a shared "interlingua" called LTML (Learnable Task Modeling Language), which integrates planning (PDDL), ontologies (OWL), and service descriptions (OWL-S).
The system utilizes a multi-strategy approach:
- Hypothesis Formers: Specialized learners like WIT (grammar induction for loops), DISTILL (causal dependency identification), and XPLAIN (explanation-based reasoning) generate partial models.
- Meta-Reasoners: A Learning Moderator manages learning goals, while the STITCHER merges overlapping partial models into a unified Hierarchical Task Network (HTN).
- Execution: The resulting model is tested using a modified SHOP2 planner (ReSHOP) on novel problem variations.
The framework was evaluated in a Medical Evacuation (Medevac) Airlift Planning domain. A single semantic trace of a human performing the task—involving airport lookups, flight scheduling, and resource reservation—served as the training data.
Results
POIROT was tested on nine variations of the Medevac problem and compared against 20 human subjects. The results indicate that POIROT's performance is remarkably close to human levels.
Key findings include:
- Accuracy: POIROT achieved ~84% accuracy in matching service calls and parameters, slightly exceeding the human average of ~81%.
- Coverage: POIROT demonstrated superior imitation of demonstration steps, reaching 98% coverage compared to the human average of ~86%.
- Goal Achievement: While humans maintained a slight edge in overall plan completion (96% vs. 86%), POIROT’s performance was statistically significant, meeting at least 85% of human performance with >99% confidence.
Discussion and Future Work
The success of POIROT highlights the power of integrated learning. No individual learner was sufficient on its own; for instance, WIT lacked branch conditions, and DISTILL missed non-causal links. Only through "stitching" did a functional model emerge.
However, limitations remain. The system currently relies heavily on semantically rich OWL-S annotations, which are not yet universal. Furthermore, POIROT is less flexible than humans when high-level strategic adjustments are required for novel scenarios. Future work will focus on "self-guided experiments" to resolve hypothesis conflicts and the full integration of additional learning components like DRILL for decision rules.
Summary updated