Abstract
Modern robotics often relies on extensive programming or massive datasets to learn new behaviors. This paper presents Rosie, an AI agent designed for Interactive Task Learning (ITL) that can acquire complex, multi-stage hierarchical tasks from a single natural language training session. By utilizing the Soar cognitive architecture and a Task Concept Network (TCN), Rosie composes physical, communicative, and mental actions into a unified framework. We demonstrate Rosie's capabilities in a simulated military barracks environment, where it successfully learns and executes a long-horizon "interior guard" task autonomously after just 17 minutes of instruction.
Introduction
The challenge of creating general-purpose robots lies in the difficulty of teaching them new tasks on the fly. Most current systems are "specialists," requiring expert intervention or thousands of demonstrations to learn a new behavior. Interactive Task Learning (ITL) seeks to bridge this gap by allowing end-users to teach robots through natural interaction. However, many existing ITL systems are limited to simple environments or struggle with complex logic such as loops, conditional branches, and mental operations. This work addresses these challenges by demonstrating an agent capable of learning a hierarchical "interior guard" task through a single interactive episode.
The Rosie Agent and Hierarchical Learning
Rosie is built upon the Soar cognitive architecture, a framework that facilitates human-like reasoning through symbolic memory and procedural rules. The learning process involves several key components:
- Situated Interactive Instruction (SII): Rosie parses natural language commands from a human instructor into precise semantic structures.
- Task Concept Network (TCN): When Rosie encounters a new task, it builds a declarative network in semantic memory that maps subtasks, goal predicates (desired end states), and control logic (loops and branches).
- Chunking: As Rosie executes a task for the first time by interpreting the TCN, it "chunks" the experience into new procedural rules. This allows the agent to transition from slow, deliberate reasoning to fast, automatic execution in future trials.
A critical feature of Rosie is its true compositionality. As shown in the task hierarchy above, high-level tasks like guard are decomposed into subtasks such as inspect, which further break down into actions like fetch or go-to-loc. This structure allows Rosie to integrate physical actions with communicative ones (e.g., asking "Who is my relieving officer?") and mental ones (e.g., remembering the answer for later use).
Experimental Evaluation
The system was evaluated in a custom simulated barracks environment containing distinct functional areas such as a motor pool, mess hall, and supply room.
The training phase consisted of a sequence of 224 individual tasks taught over a 17-minute interaction. The subsequent evaluation involved an autonomous 28-minute "guard" scenario comprising 389 individual tasks, testing the agent's ability to maintain the barracks without further human intervention.
Results and Discussion
Rosie demonstrated significant efficiency and autonomy. After the brief instruction period, the agent performed the complex patrol route autonomously for nearly half an hour without needing to ask clarifying questions. Key achievements include:
- One-Shot Generalization: Rosie learned underlying logic (e.g., "if the light is off, turn it on") rather than fixed coordinates, allowing it to adapt to different room layouts or starting conditions immediately.
- Complex Logic Handling: The agent successfully managed conditional branches (e.g., fetching a fire extinguisher only if one was missing from a sentry post) and loops (e.g., patrolling until a specific relieving officer arrived).
Conclusion
While Rosie shows promise for one-shot hierarchical learning, limitations remain. The natural language interface currently requires a somewhat structured form of English, and the simulation environment lacks the "noise" and perceptual errors of real-world robotics. Future work will focus on handling greater task variation and transitioning the system to physical robotic platforms in more complex environments.
Summary updated