Abstract
General autonomous agents must balance real-time environmental reactivity with the continuous acquisition of knowledge to improve future performance. This paper proposes a dual-level framework for learning, inspired by Daniel Kahneman’s "Thinking, Fast and Slow." We distinguish between Level 1 (L1)—automatic, innate architectural learning—and Level 2 (L2)—deliberate, knowledge-driven strategies that structure an agent's behavior to maximize what L1 can extract from experience. By implementing this framework in the Rosie robotic agent, we demonstrate how agents can learn over 30 distinct tasks through natural language interaction and transfer knowledge across domains.
Introduction
Traditionally, artificial intelligence research has treated different learning modalities—such as reinforcement learning, learning by analogy, or instruction—as isolated, special-purpose modules. This fragmented approach lacks the organizational cohesion seen in human cognition. Humans do not merely learn "automatically"; we deliberately decide to practice skills, rehearse information, or seek instruction. This paper addresses the challenge of integrating these "fast" (automatic) and "slow" (deliberate) processes into a single, cohesive architecture for general intelligence.
The L1/L2 Learning Framework
The proposed hierarchy distinguishes between the mechanisms of learning and the strategies used to invoke them:
- Level 1 (L1) - Architectural Learning: These are fixed, innate algorithms embedded in the agent's cognitive architecture. They are automatic, effortless, and operate outside the agent’s direct control. Examples include updating weights in a neural network, recording episodic memories, or reinforcement learning updates.
- Level 2 (L2) - Deliberate Learning Strategies: These are voluntary tasks the agent adopts to improve its own learning. Crucially, L2 strategies do not require new learning algorithms; instead, they utilize the agent's existing cognitive capabilities—such as planning, reasoning, and dialogue—to curate specific experiences that L1 mechanisms can then process.
Implementation: The Rosie Agent
To evaluate this framework, the authors utilized Rosie, a robotic agent implemented within the Soar cognitive architecture. Rosie learns new tasks through natural language interaction. When Rosie identifies a gap in its knowledge, it initiates a four-phase L2 process:
- Identification: Recognizing a learning opportunity (e.g., "I do not know how to solve this puzzle").
- Selection: Choosing a strategy (e.g., "I will ask the instructor for a hint").
- Execution: Interacting with the environment or instructor and replaying the experience internally.
- Utilization: Applying the newly consolidated knowledge to solve future problems.
Experimental Evaluation
Rosie was not tested on static datasets but through interactive task learning across several domains:
- Puzzles and Games: Over 30 distinct tasks, including the Tower of Hanoi.
- Robotics: Navigation and delivery tasks in a physical or simulated environment.
- Instruction: Real-time natural language processing with a human instructor.
Key Results
- Task Versatility: The L1/L2 split proved viable for general agents, allowing Rosie to learn a wide variety of tasks from scratch.
- Transfer Learning: As the agent acquired more tasks, the time required to learn subsequent tasks decreased significantly. L2 strategies enabled the agent to identify and transfer overlapping concepts and subgoals from prior experiences.
- Efficiency: While L1 mechanisms like reinforcement learning often require thousands of trials, the L2 layer allows for "fast" learning from a single instruction by synthesizing multiple knowledge sources.
Discussion and Taxonomy
A central insight of this work is that L2 is inherently meta-cognitive—it is "learning how to learn." By using complex reasoning to feed better data into basic L1 mechanisms, the agent becomes an active participant in its own development. Furthermore, many effective L2 strategies are social, suggesting that the ability to deliberately structure behavior for learning may be a key component of high-level intelligence.
The authors also propose an extended taxonomy:
- Level 0: Evolutionary changes to the architecture.
- Level 1+: Innate drives, such as curiosity.
- Level 3: Deliberate modification of the architecture itself (e.g., "brain hacking").
Limitations and Future Work
Currently, Rosie's strategy selection is limited; the agent often defaults to a single strategy (like asking for help) rather than reasoning about which L2 strategy is most efficient for a given context. Additionally, a "temporal paradox" exists: L1 mechanisms can be mathematically "slow" (requiring many samples), while L2 processing is "slow" in terms of real-time interaction. Future research will focus on optimizing strategy selection and exploring the implementation of Level 3 architectural modifications.
Summary updated