Executive Function: A Contrastive Value Policy for Resampling and Relabeling Perceptions via Hindsight Summarization?

Chris Lengerich, Benjamin J. LengericharXiv 2022 · OpenReview

Abstract

We develop the few-shot continual learning task from first principles and hypothesize an evolutionary motivation and mechanism of action for executive function as a contrastive value policy which resamples and relabels perception data via hindsight summarization to minimize attended prediction error, similar to an online prompt engineering problem. This is made feasible by the use of a memory policy and a pretrained network with inductive biases for a grammar of learning and is trained to maximize evolutionary survival. We show how this model of executive function can be used to implement hypothesis testing as a stream of consciousness and may explain observations of human few-shot learning and neuroanatomy.

Introduction

Modern machine learning excels at static tasks with massive datasets but struggles with few-shot continual learning: the ability to learn new concepts rapidly from minimal examples without "catastrophic forgetting." Humans, by contrast, efficiently distill a "firehose" of sensory data (estimated at 7GB/s for the human eye) into sparse, compositional, and causal world models. This research note proposes a formal computational mechanism for "executive function"—the brain's management system—that directs attention and memory to enable this efficiency.

The Proposed Framework: Executive Function as a Policy

The authors hypothesize that executive function acts as a contrastive value policy. It manages learning by resampling and relabeling sensory data through hindsight summarization. By treating learning as an online prompt engineering problem over a "grammar of learning," the framework provides a roadmap for agents to build causal world models from high-bandwidth streams.

The agent-environment interaction loop, highlighting the role of online learning and fitness-based deletion in an evolutionary context.

Key components of this model include:

  • Active Inference: The agent doesn't just passively observe; it takes actions (both external movements and internal shifts in attention) specifically to minimize expected prediction error.
  • Contrastive Value Policy: Executive function is defined as stateful attention to a stack of predictions and perceptions. It decides which "prediction losses"—the gap between expectation and reality—are worth attending to for model updates.

Core Mechanism: Hindsight Summarization and Relabeling

A central mechanism is Hindsight Summarization. Instead of storing raw data, the agent uses a memory policy to look back at recent trajectories and "relabel" them into abstract summaries. This reduces the dimensionality of dense sensory data into a "grammar of learning."

The hindsight summarization process, showing how trajectories are compiled into a summarized dataset to train the model across time steps.

This process allows for "fast learning" by finding transformations that map new experiences into a universal grammar, effectively "prompting" the agent's own pretrained world model. High prediction loss is experienced as cognitive dissonance, which executive function resolves by either updating the world model (learning) or taking actions to change the environment (active inference).

Theoretical Alignments and Implications

As a theoretical framework, the proposed model aligns with several observations in cognitive science and neuroanatomy:

  • Neuroanatomy: Regions associated with executive function, like the prefrontal cortex, are significantly larger in humans than other primates, correlating with superior few-shot learning capabilities.
  • The Scientific Method: The paper frames hypothesis testing, experimentation, and summary as a specific sequence of hindsight summarization steps implemented within this architecture.
  • Algorithmic Parallels: The proposed mechanism shares similarities with existing techniques like Hindsight Experience Replay (HER) and data2vec, but offers greater flexibility by operating on a faster timescale.

Conclusion

This research note provides a first-principles derivation of executive function as a solution to the few-shot continual learning problem. While empirical validation in specific neural architectures remains for future work, the framework offers a compelling explanation for how biological agents achieve sparse, causal representations from dense sensory input.

Summary updated

Abstract

We develop the few-shot continual learning task from first principles and hypothesize an evolutionary motivation and mechanism of action for executive function as a contrastive value policy which resamples and relabels perception data via hindsight summarization to minimize attended prediction error, similar to an online prompt engineering problem. This is made feasible by the use of a memory policy and a pretrained network with inductive biases for a grammar of learning and is trained to maximize evolutionary survival. We show how this model of executive function can be used to implement hypothesis testing as a stream of consciousness and may explain observations of human few-shot learning and neuroanatomy.
BibTeX
@misc{lengerich2022executive,
  author = {Chris Lengerich and Benjamin J. Lengerich},
  title = {{Executive Function: A Contrastive Value Policy for Resampling and Relabeling Perceptions via Hindsight Summarization?}},
  year = {2022},
  eprint = {2204.12639},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.12639},
}
Figure