Exploratory Learning Strucutre in Artificial Cognitive Systems

Michael Felsberg, Johan Wiklund, Erik Jonsson, Anders Moe, Gösta H. GranlundKTH 2019 · DOI

TL;DR

This paper introduces the COSPAL project's architecture for artificial cognitive systems, focusing on "exploratory learning." Unlike traditional methods that aim for generalization within a known class, COSPAL's system learns to apply existing skills to new problem classes through incremental learning and a novel four-loop feedback structure. This approach, heavily reliant on user interaction for guidance, aims to create more robust and adaptable AI systems.

What Problem Does This Paper Address?

The paper tackles the challenge of creating artificial cognitive systems that can go beyond simply mastering a predefined set of tasks. Current machine learning often excels at generalization within a known domain but struggles to adapt to entirely new situations or problem classes. The authors propose "exploratory learning" as a strategy to overcome this limitation. This involves not just learning from data but actively applying existing knowledge to novel scenarios, requiring incremental learning and a flexible architecture. A key aspect is shifting the burden of defining the system's world model from the designer to the user, fostering a more interactive and adaptable learning process.

Methods

The core of the COSPAL architecture is a layered system with three main units:

  • Perception-Action (PA) Unit: Handles low-level control cycles and perception-action loops.
  • Grounding & Management (GM) Unit: Manages and selects models, mapping sub-symbolic information to simple symbolic representations.
  • Symbolic Processing (SP) Unit: Responsible for concept generation, supervision, and high-level control, relating symbols to each other and to symbolic tasks.

These units interact through a system of nested feedback loops. The outermost loop involves user reinforcement, while inner loops manage symbolic and sub-symbolic processing and perception-action cycles. This layered approach allows for incremental and exploratory learning. The paper also details a visual recognition module based on "P-channels" and sampled kernel density estimation, which is designed to be robust to clutter and missing information. Bootstrapping, a process of initial intensive learning, is also discussed as a way to quickly bring the system to a useful state.

Datasets

Not discussed in the paper. The paper mentions using the COIL-100 database for evaluating the visual recognition method, noting its performance is nearly perfect for simple scenarios.

Key Insights and Ideas

  • Exploratory Learning: The central concept is moving beyond "best possible generalization" within a class to applying learned competences to new problem classes. This is crucial for creating truly adaptable AI.
  • User-Centric Bootstrapping and Learning: The system's learning is guided by user-defined scenarios and feedback, rather than relying on a pre-programmed world model. This interactive approach is seen as key to achieving robust and extendable behavior.
  • Nested Feedback Loops: The four-level hierarchy of feedback loops (user, symbolic, sub-symbolic, perception-action) is designed to enable incremental learning and allow for different levels of complexity.
  • Separation of Concerns: The three distinct units (PA, GM, SP) reflect a hierarchy of information processing (sub-symbolic, first-order symbolic, relations between symbols), which is essential for hierarchical learning.
  • P-Channels for Visual Recognition: The use of P-channels and kernel density estimation for visual recognition is highlighted for its robustness to background clutter and missing information, even in complex scenes.

Key Results

The paper presents preliminary results indicating that the proposed architecture concept is successful. While no specific quantitative results are provided for the overall system, the visual recognition module is noted to perform nearly perfectly on the COIL-100 dataset. The authors state that the available preliminary results clearly point towards a successful proof of the architecture concept.

Limitations and Future Work

The authors acknowledge that the described system is a work in progress and no final results are available yet. Future work will focus on:

  • Virtually removing the manipulator from the visual recognition process.
  • Improving the detection of candidate regions for recognition.
  • Evaluating the subspace projection method for generalizing object properties.
  • Integrating the various modules into a fully closed system, as current results are often from single modules or off-line data exchange.
  • Further experiments and publications are planned.

Summary updated

Abstract

One major goal of the COSPAL project is to develop an artificial cognitive system architecture with the capability of exploratory learning. Exploratory learning is a strategy that allows to apply generalization on a conceptual level, resulting in an extension of competences. Whereas classical learning methods aim at best possible generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class. Incremental or online learning is an inherent requirement to perform exploratory learning. Exploratory learning requires new theoretic tools and new algorithms. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning and in this paper we focus on its algorithmic aspect. Learning is performed in terms of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail. We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user ('teacher') and system is a major difference to many existing systems where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems. We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module. We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.
BibTeX
@inproceedings{felsberg2019exploratory,
  author = {Michael Felsberg and Johan Wiklund and Erik Jonsson and Anders Moe and Gösta H. Granlund},
  title = {{Exploratory Learning Strucutre in Artificial Cognitive Systems}},
  booktitle = {KTH},
  year = {2019},
  doi = {10.2390/biecoll-icvs2007-173},
}
Figure