Interactively Learning a Blend of Goal-Based and Procedural Tasks

Aaron Mininger, John E. LairdAAAI 2018 · DOI

Introduction

In the field of Interactive Task Learning (ITL), agents are typically taught using one of two paradigms: goal-based or procedural. Goal-based learning involves defining a desired end state (e.g., "the soda is in the garbage"), allowing the agent flexibility in planning but requiring a robust internal model of world physics. Conversely, procedural learning provides a step-by-step script (e.g., "pick up soda, go to garbage, drop soda"), which is easier to teach but often brittle when environmental conditions change. This paper introduces a hybrid approach implemented in the Rosie agent, built on the Soar cognitive architecture, which allows for a seamless blend of these two methods.

Task Representation and the Fallback Mechanism

The core innovation of this research is the expanded Task Concept Network (TCN). While traditional TCNs represent tasks as single goals, the authors expanded them to support an ordered list of subgoals. These subgoals can be state-based predicates or references to other procedural subtasks.

The Task Concept Network for the 'discard' task, illustrating the integration of goal structures and subtask sequences.

Rosie utilizes Retrospective Policy Learning through Explanation-Based Learning (EBL) to generalize rules from its episodic memory. A critical feature is the "fallback" mechanism: if the agent lacks the domain knowledge to explain why an action led to a goal (for instance, if it doesn't understand the mechanics of a 'pickup' action), it does not fail. Instead, it creates a procedural subgoal to execute that specific action at that point in the sequence, effectively "memorizing" the step while maintaining goal-oriented reasoning for the rest of the task.

Experimental Environment and Evaluation

The system was evaluated in a simulated multi-room office environment. The agent perceives the world through symbolic representations, mapping visual entities to logical predicates that describe their properties and spatial relationships.

The simulated environment alongside the symbolic predicates used by the agent to represent the world state.

Two primary case studies demonstrated the system's versatility:

  1. The "Tour" Task: A purely procedural task where the agent navigates to locations and recites phrases. Because the world state remains unchanged, there is no "goal state," and the agent successfully learns the task as a sequence of procedural subgoals.
  2. The "Deliver" Task: To test the fallback mechanism, researchers removed the agent's internal model for the pickup action. Rosie successfully learned a hybrid policy, using a procedural step for the pickup and resuming goal-based planning for the delivery phase.

Key Insights and Results

The research demonstrates that an agent does not require a perfect world model to be effective. By mixing goals and procedures, Rosie exhibits "graceful degradation of knowledge," using planning where it has expertise and rote memorization where it is ignorant. The system is highly efficient, capable of single-shot generalization—learning a complex hierarchical task from one example and immediately applying it to new objects or locations.

Limitations and Future Work

While successful in sequential tasks, the current system faces challenges with complex control flows, such as loops ("pick up all trash") or conditional branching. Furthermore, the agent is currently limited to singular entities and cannot yet reason about collections or numeric quantities. Future work will focus on allowing instructors to edit learned procedures and expanding the agent's ability to learn new primitive motor skills through linguistic interaction.

Summary updated

BibTeX
@inproceedings{mininger2018interactively,
  author = {Aaron Mininger and John E. Laird},
  title = {{Interactively Learning a Blend of Goal-Based and Procedural Tasks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11488},
}
Figure