Human Curriculum Effects Emerge with In-Context Learning in Neural Networks

Jacob Russin, Ellie Pavlick, Michael J. FrankarXiv 2024 · DOI · arXiv

Introduction

Researchers at Brown University have identified a significant link between artificial intelligence architectures and human cognitive processes. Their study explores how neural networks utilize two distinct learning systems—fast, activation-based In-Context Learning (ICL) and slow, weight-based In-Weight Learning (IWL)—to produce behaviors that mirror human learning patterns. Specifically, this dual-process framework explains why humans exhibit varying performance based on whether information is presented in grouped (blocked) or shuffled (interleaved) sequences.

The Dual-Process Learning Framework

In human psychology, learning is often categorized into rapid, rule-based inference and slow, incremental habit formation. Modern Transformer models replicate this duality: they can learn new tasks from immediate context (ICL) or through permanent weight adjustments via backpropagation (IWL).

Comparison of In-Context Learning (ICL) and In-Weight Learning (IWL) dynamics.

The study investigates how these systems interact, revealing that ICL tends to favor "blocked" curricula where categories are grouped, while IWL benefits from "interleaved" curricula that prevent forgetting.

Methods and Experimental Design

The researchers utilized Transformer-based networks trained through metalearning to develop ICL capabilities. These models were then tested on synthetic tasks designed to simulate human psychology experiments:

  • Category-Learning: Categorizing items based on features like line length and orientation.
  • Compositional Mapping: Mapping color-animal pairs (e.g., "blue alligator") to specific coordinates.

The experiments compared "Rule-like" conditions, where a single feature determines the outcome, with "Rotated" conditions requiring the integration of multiple complex features.

Key Insights: Suppression and Curriculum Effects

A central finding is the "Suppression" Effect. When ICL successfully identifies a pattern, it reduces the error signal that drives IWL. Consequently, rapid learning through context can actually hinder the long-term "baking" of information into the model's weights.

  • Blocking Advantage: Grouping similar examples makes underlying rules more salient, allowing the attention mechanism (ICL) to quickly grasp the task.
  • Interleaving Advantage: Shuffling examples is necessary for IWL to avoid "catastrophic forgetting," where the model over-optimizes for the most recent category at the expense of previous ones.

Results and the Flexibility-Retention Tradeoff

The models successfully replicated human curriculum effects, showing a blocking advantage for simple rules and an interleaving advantage for complex, integrated tasks.

Experimental results comparing IWL-only models with combined IWL+ICL models across different curricula.

The study also uncovered a fundamental Flexibility-Retention Tradeoff. Models with high "working memory" (strong ICL) were exceptionally fast at adapting to new rules but showed poor long-term retention. Conversely, when cognitive load was simulated by limiting the model's attention, the networks were slower to learn but demonstrated superior retention because they were forced to rely on more permanent IWL updates.

Conclusion

This research demonstrates that the interplay between ICL and IWL in neural networks provides a robust model for understanding human curriculum effects. While Transformers are not perfect biological analogs, they capture the essential dynamics of dual-process learning. Future research will focus on how these dynamics scale to high-dimensional reasoning and the specific inductive biases that lead Large Language Models to develop these human-like learning preferences.

Summary updated

Abstract

Human learning embodies a striking duality: sometimes, we appear capable of following logical, compositional rules and benefit from structured curricula (e.g., in formal education), while other times, we rely on an incremental approach or trial-and-error, learning better from curricula that are randomly interleaved. Influential psychological theories explain this seemingly disparate behavioral evidence by positing two qualitatively different learning systems -- one for rapid, rule-based inferences and another for slow, incremental adaptation. It remains unclear how to reconcile such theories with neural networks, which learn via incremental weight updates and are thus a natural model for the latter type of learning, but are not obviously compatible with the former. However, recent evidence suggests that metalearning neural networks and large language models are capable of "in-context learning" (ICL) -- the ability to flexibly grasp the structure of a new task from a few examples. Here, we show that the dynamic interplay between ICL and default in-weight learning (IWL) naturally captures a broad range of learning phenomena observed in humans, reproducing curriculum effects on category-learning and compositional tasks, and recapitulating a tradeoff between flexibility and retention. Our work shows how emergent ICL can equip neural networks with fundamentally different learning properties that can coexist with their native IWL, thus offering a novel perspective on dual-process theories and human cognitive flexibility.
BibTeX
@misc{russin2024human,
  author = {Jacob Russin and Ellie Pavlick and Michael J. Frank},
  title = {{Human Curriculum Effects Emerge with In-Context Learning in Neural Networks}},
  year = {2024},
  eprint = {2402.08674},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.08674},
}
Figure