Abstract
It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could 1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, 2) capture human data on curriculum effects in schema learning, and 3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.
Introduction
Humans possess a remarkable ability to segment continuous experience into meaningful events. This process, known as event segmentation, is thought to rely on the inference of "latent causes" (LCs)—hidden variables that explain the structure of the environment. For instance, when learning two similar languages like Norwegian and Swedish, the brain must distinguish between context-specific vocabulary (e.g., "water" as vann vs. vatten) while simultaneously leveraging shared structure (e.g., "book" as bok in both).
Existing computational models, such as Structured Event Memory (SEM), address this by "splitting" every new context into a separate neural network. While effective at preventing catastrophic interference, this approach is biologically implausible and computationally expensive, as it fails to share knowledge across contexts and requires an ever-expanding set of parameters. This paper introduces the Latent Cause Network (LCNet), a model designed to reconcile the need for specialization with the benefits of generalization.
The LCNet Architecture
LCNet departs from the "one network per task" paradigm by utilizing a single shared network for all experiences. To maintain context-specific knowledge without interference, it employs three key components:
- Shared Network Weights: All tasks are learned within a common set of weights, facilitating the extraction of structure common to multiple latent causes.
- Random High-Dimensional Context Vectors: Each inferred LC is assigned a unique, random vector. These vectors are approximately orthogonal in high-dimensional space and serve as "keys" that modulate the hidden layer activity, effectively shifting the network into a task-specific state.
- Bayesian Nonparametric Inference: The model uses a "sticky Chinese Restaurant Process" (sCRP) to determine whether current observations belong to an existing LC or necessitate the creation of a new one.
- Episodic Memory Shortcut: An optional key-value buffer allows the model to quickly retrieve context vectors once an LC is identified, bypassing expensive Bayesian calculations for every step.
This architecture allows the model to factorize knowledge into shared terms (stored in weights) and idiosyncratic terms (modulated by the context vector).
Experimental Results
The researchers validated LCNet across three distinct domains:
1. Synthetic Function Learning
In a regression task involving polynomial functions composed of shared and idiosyncratic components, LCNet demonstrated superior efficiency. Unlike SEM, which must learn each function from scratch, LCNet reused the shared structure captured in its weights, leading to faster convergence on new tasks. By "lesioning" the context vector pathway, the authors confirmed that the shared structure was indeed stored in the network's core weights.
2. Narrative Event Schemas
LCNet was tested on its ability to learn state-transition schemas (e.g., navigating a cafe vs. a bar). The model successfully replicated human behavioral data showing that "blocked" learning (focusing on one schema at a time) can be more effective than "interleaved" learning for complex schemas. Standard recurrent neural networks (RNNs) typically fail this test due to catastrophic forgetting, whereas LCNet's "sticky" prior allows it to maintain context stability.
3. Naturalistic Event Segmentation
Finally, the model was applied to the META corpus, a dataset of 25 hours of daily activity videos. LCNet's inferred event boundaries—the points where it switched latent causes—correlated significantly ($p < 0.00001$) with human-annotated event boundaries. This suggests that the model's principles of latent cause inference scale effectively from controlled laboratory tasks to complex, real-world environments.
Discussion and Future Work
LCNet provides a computationally feasible and biologically plausible framework for understanding how the brain manages the trade-off between specialization and generalization. By using high-dimensional representations to index contexts within a shared network, it avoids the parameter explosion of previous models while maintaining high performance.
Future work aims to address current limitations, such as the assumption of independent, random contexts. Real-world contexts are often hierarchical or overlapping (e.g., "making coffee" while "talking on the phone"). Developing a learned "feature space" for contexts where similar tasks share similar vectors would further enhance the model's ability to generalize across related experiences.
Summary updated