Abstract
We develop the few-shot continual learning task from first principles and hypothesize an evolutionary motivation and mechanism of action for executive function as a contrastive value policy which resamples and relabels perception data via hindsight summarization to minimize attended prediction error, similar to an online prompt engineering problem. This is made feasible by the use of a memory policy and a pretrained network with inductive biases for a grammar of learning and is trained to maximize evolutionary survival. We show how this model of executive function can be used to implement hypothesis testing as a stream of consciousness and may explain observations of human few-shot learning and neuroanatomy.
Introduction
Modern machine learning excels at static tasks with massive datasets but struggles with few-shot continual learning: the ability to learn new concepts rapidly from minimal examples without "catastrophic forgetting." Humans, by contrast, efficiently distill a "firehose" of sensory data (estimated at 7GB/s for the human eye) into sparse, compositional, and causal world models. This research note proposes a formal computational mechanism for "executive function"—the brain's management system—that directs attention and memory to enable this efficiency.
The Proposed Framework: Executive Function as a Policy
The authors hypothesize that executive function acts as a contrastive value policy. It manages learning by resampling and relabeling sensory data through hindsight summarization. By treating learning as an online prompt engineering problem over a "grammar of learning," the framework provides a roadmap for agents to build causal world models from high-bandwidth streams.
Key components of this model include:
- Active Inference: The agent doesn't just passively observe; it takes actions (both external movements and internal shifts in attention) specifically to minimize expected prediction error.
- Contrastive Value Policy: Executive function is defined as stateful attention to a stack of predictions and perceptions. It decides which "prediction losses"—the gap between expectation and reality—are worth attending to for model updates.
Core Mechanism: Hindsight Summarization and Relabeling
A central mechanism is Hindsight Summarization. Instead of storing raw data, the agent uses a memory policy to look back at recent trajectories and "relabel" them into abstract summaries. This reduces the dimensionality of dense sensory data into a "grammar of learning."
This process allows for "fast learning" by finding transformations that map new experiences into a universal grammar, effectively "prompting" the agent's own pretrained world model. High prediction loss is experienced as cognitive dissonance, which executive function resolves by either updating the world model (learning) or taking actions to change the environment (active inference).
Theoretical Alignments and Implications
As a theoretical framework, the proposed model aligns with several observations in cognitive science and neuroanatomy:
- Neuroanatomy: Regions associated with executive function, like the prefrontal cortex, are significantly larger in humans than other primates, correlating with superior few-shot learning capabilities.
- The Scientific Method: The paper frames hypothesis testing, experimentation, and summary as a specific sequence of hindsight summarization steps implemented within this architecture.
- Algorithmic Parallels: The proposed mechanism shares similarities with existing techniques like Hindsight Experience Replay (HER) and data2vec, but offers greater flexibility by operating on a faster timescale.
Conclusion
This research note provides a first-principles derivation of executive function as a solution to the few-shot continual learning problem. While empirical validation in specific neural architectures remains for future work, the framework offers a compelling explanation for how biological agents achieve sparse, causal representations from dense sensory input.
Summary updated