Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes

Qihong Lu, Tan T Nguyen, Qiong Zhang, Uri Hasson, Thomas L. Griffiths, Jeffrey M. Zacks, Samuel J. Gershman, Kenneth A. Norman

Abstract

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could 1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, 2) capture human data on curriculum effects in schema learning, and 3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.

Introduction: The Lumping vs. Splitting Dilemma

When humans process a stream of experiences, they naturally partition them into distinct events or "latent causes." A significant challenge in cognitive modeling is balancing the learning of general rules (shared structure) with context-specific details. For example, learning two similar languages requires leveraging shared vocabulary while avoiding "catastrophic interference," where learning a word in one language causes the learner to forget its counterpart in another.

Previous models, such as Structured Event Memory (SEM), addressed interference by creating entirely new neural networks for every new context. While this prevents interference, it is computationally expensive and makes knowledge sharing impossible. LCNet was developed to solve this dilemma, aiming for a model that is both neurally plausible and scalable.

Conceptual framework of latent cause inference and event boundaries.

Methods: The LCNet Architecture

LCNet introduces an architectural middle ground between total integration and total separation. It utilizes a shared backbone—a single neural network—to capture structure common across all tasks. To handle context-specific details, the model assigns a unique, high-dimensional random context vector to each inferred latent cause. This vector is fed into the network's hidden layer, effectively shifting activity into a specific region of its state space.

The model employs a Bayesian nonparametric algorithm (a "sticky" Chinese Restaurant Process) to decide on the fly whether an experience belongs to a known context or requires a new "split." Additionally, an episodic memory buffer acts as a shortcut, allowing the model to quickly recognize familiar contexts without re-running complex Bayesian calculations.

LCNet architecture and performance on the polynomial function learning task.

Results and Key Insights

The researchers validated LCNet across three environments: synthetic polynomial function learning, a Cafe/Bar schema based on human behavioral studies, and the naturalistic META video corpus.

Knowledge Factorization and Orthogonality

By "lesioning" (disabling) the context module after training, the researchers found the network could still produce the "shared" components of a task. This proves the model successfully factorized its knowledge: the main weights learned general rules, while the context module learned exceptions. This is made possible by the mathematical property of high-dimensional space, where random vectors are almost always orthogonal, preventing different contexts from interfering with each other.

Human Alignment and Scalability

LCNet successfully replicated human performance patterns in schema learning, specifically showing that "blocked" training (learning one context at a time) leads to better schema acquisition than interleaved training—a scenario where standard neural networks typically fail due to interference. Furthermore, when applied to the META video dataset, LCNet’s self-identified event boundaries showed a significant correlation ($p < 0.00001$) with human segmentation of the same videos.

Discussion and Future Directions

LCNet demonstrates a computationally feasible approach to reconciling shared and context-specific structure. While successful, the authors note that future versions could move beyond random context vectors to a learned "feature space" where related contexts (e.g., a Cafe and a Bar) have similar vectors. Additionally, integrating LC inference into Transformer-based Large Language Models (LLMs) could help reduce "hallucinations" caused by improper context blending, leading to more human-like event cognition.

Summary updated

Abstract

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could 1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, 2) capture human data on curriculum effects in schema learning, and 3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.
BibTeX
@inproceedings{lu2024reconciling,
  author = {Qihong Lu and Tan T Nguyen and Qiong Zhang and Uri Hasson and Thomas L. Griffiths and Jeffrey M. Zacks and Samuel J. Gershman and Kenneth A. Norman},
  title = {{Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-64272-5},
}

Loading related content...

Figure