Generative Environment-Representation Instance-Based Learning: A Cognitive Model

Tyler Malloy, Yinuo Du, Fei Fang, Cleotilde GonzálezAAAI 2024 · DOI

This paper, presented at the AAAI Fall Symposium Series (FSS-23), introduces a novel framework that bridges the gap between deep generative modeling and classical cognitive science.

TL;DR

The authors propose GERIBL (Generative Environment-Representation Instance-Based Learning), a model that uses Artificial Neural Networks to automatically learn how to represent complex environments for cognitive decision-making. By integrating generative models like VAEs and GANs with Instance-Based Learning Theory, they eliminate the need for hand-crafted features and create a more scalable, human-like model for dynamic tasks.

What Problem Does This Paper Address?

Instance-Based Learning Theory (IBLT) is a well-established framework for modeling how humans make decisions by retrieving similar past experiences (instances) from memory. However, IBLT has a "scaling problem."

In simple tasks, it’s easy to define the features of a situation (e.g., "the light is red"). But in complex, high-dimensional environments—like navigating a city or interpreting medical images—defining which features matter is incredibly difficult. Traditionally, researchers have had to "hand-craft" these features, which is time-consuming, prone to bias, and fails to generalize to new tasks. The authors address this by asking: Can we use generative AI to "see" and represent the environment for us, much like the human visual system does?

Methods

The authors introduce GERIBL (pronounced "jur-bl"), which replaces the manual feature-engineering step with a Generative Model (GM).

The workflow is as follows:

  1. Representation Learning: A GM (like a Variational Autoencoder) is pre-trained on environment stimuli to learn a low-dimensional "latent code" ($z$) that captures the essence of the scene.
  2. Memory Storage: Instead of storing raw data, the IBL model stores these latent codes as part of its experience triplets (state, action, outcome).
  3. Similarity-Based Retrieval: When faced with a new situation, the model calculates similarity within this learned latent space to retrieve relevant memories.

The researchers tested several GM architectures:

  • Variational Autoencoders (VAEs) and $\beta$-VAEs: Models that learn to compress and reconstruct images.
  • Generative Adversarial Networks (GANs): Models that learn by trying to "fool" a discriminator.
  • Vision Transformers (ViT): Specifically looking at "Attention" values as a way to represent what the model finds important.

Datasets

The model was evaluated using data from two human behavioral experiments:

  • Experiment 1 (Visual Utility Learning): A contextual 3-armed bandit task where participants had to learn which combinations of shapes, colors, and textures yielded the highest rewards.
  • Experiment 2 (Transfer of Learning): A task designed to see how well participants (and models) could apply knowledge from simple stimuli (just shapes) to more complex ones (shapes + colors + textures).

Key Insights and Ideas

  • The "Less is More" Bottleneck: One of the most striking findings was that GMs with smaller representation sizes (more compressed) actually fit human behavior better than high-capacity models. This suggests that the human cognitive "bottleneck"—our inability to process every single detail—is actually a feature that helps us generalize and learn faster.
  • Attention as a Proxy for Cognition: In transfer-of-learning tasks, the Transformer model using "Attention" values performed exceptionally well. It suggests that the internal mechanisms AI uses to "focus" on parts of an image may align closely with human selective attention.
  • Closing the Loop: The authors suggest that representations shouldn't be static. By using the "blending" error from the decision model to fine-tune the generative model, the system can learn to prioritize features that are actually relevant to winning the game, mimicking how humans learn to ignore distractions.

Key Results

  • Human-Like Performance: All tested generative models were able to roughly emulate human learning curves in the bandit tasks.
  • Superior Generalization: The GERIBL models generally showed a higher "Transfer of Learning" capability compared to standard IBL models using hand-crafted features.
  • Best in Class: The Transformer-based Attention model was the top performer in generalization, matching human performance regardless of the specific stimuli it was originally trained on.

Limitations and Future Work

While GERIBL is a significant step forward, the authors note a few areas for growth:

  • Stimulus Complexity: The experiments used relatively simple geometric shapes. Testing the model on "natural" images (real-world photos) remains a hurdle for future research.
  • Biological Plausibility: While $\beta$-VAEs have some links to biological visual processing, other models like GANs are less "brain-like." Future work could explore more biologically inspired generative architectures.
  • Integration: The authors aim to eventually integrate GERIBL into broader cognitive architectures like ACT-R to provide a more holistic model of human intelligence.

Summary updated

Abstract

Instance-Based Learning Theory (IBLT) suggests that humans learn to engage in dynamic decision making tasks through the accumulation of experiences, represented by the decision task features, the actions performed, and the utility of decision outcomes. This theory has been applied to the design of Instance-Based Learning (IBL) models of human behavior in a variety of contexts. One key feature of all IBL model applications is the method of accumulating instance-based memory and performing recognition-based retrieval. In simple tasks with few features, this knowledge representation and retrieval could hypothetically be done using all relevant information. However, these methods do not scale well to complex tasks when exhaustive enumeration of features is unfeasible. This requires cognitive modelers to design task-specific representations of state features, as well as similarity metrics, which can be time consuming and fail to generalize to related tasks. To address this issue, we leverage recent advancements in Artificial Neural Networks, specifically generative models (GMs), to learn representations of complex dynamic decision making tasks without relying on domain knowledge. We evaluate a range of GMs in their usefulness in forming representations that can be used by IBL models to predict human behavior in a complex decision making task. This work connects generative and cognitive models by using GMs to form representations and determine similarity.
BibTeX
@inproceedings{malloy2024generative,
  author = {Tyler Malloy and Yinuo Du and Fei Fang and Cleotilde González},
  title = {{Generative Environment-Representation Instance-Based Learning: A Cognitive Model}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaaiss.v2i1.27696},
}

Loading related content...

Figure