Reconciling shared versus context-specific information in a neural network model of latent causes

Qihong Lu, Tan T Nguyen, Qiong Zhang, Uri Hasson, Thomas L. Griffiths, Jeffrey M. Zacks, Samuel J. Gershman, Kenneth A. Norman

Abstract

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could (1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, (2) capture human data on curriculum effects in schema learning, and (3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.

Introduction

In cognitive science, the theory of Latent Cause Inference (LCI) suggests that humans partition a continuous stream of experience into discrete "causes" or events to help us learn and generalize. For example, if you are learning Norwegian and Swedish, your brain needs to keep the languages separate to avoid confusion (context-specific) while recognizing that many words and grammatical structures are identical (shared structure).

Conceptual diagram of event boundaries and task switching in an agent-environment interaction.

Existing computational models, like Structured Event Memory (SEM), handle this by "splitting" off a brand-new neural network for every new context. While this prevents catastrophic interference (forgetting old tasks when learning new ones), it lacks scalability and fails to support transfer learning because knowledge is not shared between the separate networks. LCNet aims to reconcile this "sharing vs. splitting" tension within a single, scalable architecture.

Methods: The LCNet Architecture

LCNet introduces a hybrid approach to knowledge representation:

  • Shared Weights: A single neural network (either feedforward or a Recurrent Neural Network/GRU) processes all observations. This allows the network to store "universal" rules in its weights.
  • Context Vectors: To keep contexts distinct, the model uses a "context module." When a new latent cause (LC) is inferred, the model generates a unique, random high-dimensional vector. Because high-dimensional random vectors are approximately orthogonal, they act as distinct keys that prevent different tasks from interfering with one another.
  • Bayesian Inference: The model uses a sticky Chinese Restaurant Process (sCRP)—a nonparametric Bayesian algorithm—to decide whether a current observation belongs to a known context or if it is time to create a new one.
LCNet architecture and simulation results demonstrating the factorization of shared and context-specific knowledge.

Results

Simulation 1: Factorization of Knowledge

In a function-learning task involving synthetic polynomials, researchers found that LCNet successfully factorized experience. By "lesioning" (disconnecting) the context module after training, the model still produced the "shared" part of a task. This proves the main weights learn the "rules of the world," while the context vectors handle idiosyncratic exceptions. Consequently, LCNet learned new functions significantly faster than the older SEM model by reusing acquired shared knowledge.

Simulation 2: Human-Like Curriculum Effects

LCNet successfully replicated human data on how the order of learning (curriculum) affects memory. Humans often learn better when tasks are "blocked" (practicing one thing for a long time) rather than "interleaved." LCNet explains this through its "stickiness" prior—the assumption that the world doesn't change contexts every second. When the model's internal stickiness matches the environment, it segments events more accurately and avoids the catastrophic interference that plagues standard neural networks.

Simulation 3: Naturalistic Scaling

When applied to the META Corpus (a dataset of 25 hours of daily activity videos), LCNet’s inferred event boundaries showed a significant correlation ($p < 0.00001$) with human event segmentation. The model successfully identified the start and end of complex activities, such as "making breakfast," in a manner consistent with human observers.

Discussion

LCNet demonstrates a computationally feasible approach to reconciling shared and context-specific structure. However, limitations remain. Currently, LCNet uses random context vectors, meaning it does not recognize similarities between related tasks (e.g., "making coffee" vs. "making tea"). Future work will focus on learning a structured feature space for contexts and integrating episodic memory to allow the model to retrieve previous contexts more efficiently based on sensory cues.

Summary updated

Abstract

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could (1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, (2) capture human data on curriculum effects in schema learning, and (3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.
BibTeX
@inproceedings{lu2024reconciling,
  author = {Qihong Lu and Tan T Nguyen and Qiong Zhang and Uri Hasson and Thomas L. Griffiths and Jeffrey M. Zacks and Samuel J. Gershman and Kenneth A. Norman},
  title = {{Reconciling shared versus context-specific information in a neural network model of latent causes}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-64272-5},
}

Loading related content...

Figure