TL;DR
This paper introduces a novel approach for interactive task learning (ITL) where an agent, Rosie, learns to understand and execute tasks through natural language instructions. By learning hierarchical symbolic representations instead of direct mappings, Rosie can effectively disambiguate ambiguous language, transfer knowledge between tasks, and even debug its own understanding, leading to significantly faster learning.
What Problem Does This Paper Address?
The core challenge this paper tackles is enabling agents to learn new tasks quickly and efficiently from natural language instructions, a process known as Interactive Task Learning (ITL). Traditional ITL systems often struggle with:
- Ambiguity: Natural language is inherently ambiguous. Words can have multiple meanings (polysemy), and the same meaning can be expressed by different words.
- Lack of Common Ground: The agent and instructor may not share the same understanding of the world or task concepts.
- Distractors: Environments often contain irrelevant information that can confuse the learning process.
- Knowledge Interference: When learning multiple tasks, knowledge from previous tasks can sometimes hinder the learning of new ones (negative transfer).
Existing approaches often assume one-to-one mappings between words and task components or rely on massive training data for single tasks. This paper aims to overcome these limitations by enabling an agent to learn from a sequence of tasks, transfer knowledge effectively, and resolve ambiguities through interaction.
Methods
The paper presents an extension to Rosie, an ITL agent built on the Soar cognitive architecture. The key innovation lies in learning hierarchical symbolic representations rather than direct perceptual-to-action mappings. This approach involves:
- Hierarchical Symbolic Representations: Instead of learning a direct mapping from words to actions or states, Rosie learns structured, symbolic representations of task elements (goals, actions, terms). These representations are built hierarchically, allowing for composition and reuse.
- Recursive Grounding Function (RGF): This algorithm is central to learning and applying task elements. It recursively breaks down task elements into their constituent predicates and functions, attempting to ground them in the agent's perception of the world. If a definition is missing, it prompts the teacher.
- Multiple Interpretation Generation: To handle ambiguity, Rosie generates multiple possible recognition structures (interpretations) for a given instruction, based on known meanings of words and task elements.
- Disambiguation Strategies: When multiple interpretations are possible, Rosie employs a series of strategies:
- Analyzing Task Element Outcomes: It checks if different interpretations lead to different numbers of results for the highest-level task element (goal, action, failure).
- Examining Supporting Task Elements: If no difference is found at the top level, it examines differences in the outcomes of supporting task elements.
- Requesting New Demonstrations: If all else fails, Rosie asks the teacher for a new state demonstration to help differentiate interpretations.
- Efficient Communication: Rosie uses its analysis of interpretations to ask targeted, disambiguating questions to the instructor, facilitating efficient resolution of ambiguity.
Datasets
The paper evaluates its approach on 40 common games and puzzles. These tasks were chosen to provide a large and varied set of problems with overlapping concepts and potential for both positive and negative transfer. The authors created teaching scripts that simulate a human instructor, ensuring that previously learned concepts are not re-taught.
Key Insights and Ideas
- Symbolic Hierarchies are Key to Transfer: Learning hierarchical symbolic representations, rather than direct mappings, is crucial for enabling robust knowledge transfer and composition across tasks. This allows the agent to reuse learned concepts in new contexts.
- Proactive Ambiguity Resolution: The agent's ability to proactively generate multiple interpretations and then use targeted questions to resolve ambiguity is a significant step beyond passive learning. This mirrors how humans learn and resolve misunderstandings.
- Handling Many-to-Many Mappings: The approach explicitly addresses the challenge of words having multiple meanings and multiple words referring to the same concept, a common issue in natural language.
- Debugging Own Understanding: The ability to analyze and debug its own interpretations allows the agent to identify and correct errors in its learned knowledge, leading to more accurate task execution.
- Efficiency Through Interaction: By asking precise questions, the agent minimizes the need for extensive trial-and-error or large amounts of data, making learning much faster.
Key Results
The paper demonstrates significant improvements in learning efficiency and knowledge transfer:
- Across-Task Transfer: When learning sequences of the 40 games, the number of words required to teach a new game decreased as more related games were previously taught. This indicates substantial across-task transfer.
- Within-Task Transfer: The approach supports within-task transfer, as seen in the Tic-Tac-Toe example where learning the concept of "three in a row" can be applied to variations of the game.
- Reduced Learning Time: For games with substantial conceptual overlap (e.g., Sudoku variants), learning time (measured in words) was dramatically reduced, sometimes by more than a factor of three.
- Handling Interference: The disambiguation strategies were effective in resolving ambiguous word meanings and environmental distractors, preventing negative transfer.
- Correct Generalization: The agent correctly generalized, disambiguated, and transferred concepts across variations in language descriptions and world representations.
Limitations and Future Work
The authors acknowledge several limitations:
- Perceptual Attention: The current system lacks perceptual attention mechanisms, making it computationally expensive to process states with a large number of objects and relations (e.g., Chess).
- Language Rigidity: While Rosie's language understanding is sufficient for the evaluated games, it is described as more rigid than natural language.
Future work includes:
- Exploring Additional Disambiguation Strategies: Investigating strategies that involve asking questions directly about learned representations.
- Learning Heuristics and Reward Functions: Extending the agent's learning capabilities to include these aspects.
- Expanding to Other Task Types: Applying the approach to a broader range of tasks beyond games and puzzles.
- Developing Perceptual Attention: Addressing the computational cost of processing complex states.
Summary updated