Computational models of episodic-like memory in food-caching birds

Johanni Brea, Nicola S. Clayton, Wulfram GerstnerNature Communications 2023 · DOI

TL;DR

Researchers have developed a computational model that explains the complex "episodic-like" memory of food-caching birds using associative reinforcement learning. The study demonstrates that sophisticated behaviors—like remembering what, where, and when food was hidden—can be achieved through neural mechanisms like memory consolidation and motivational control, without requiring the high-level "mental time-travel" often attributed to humans.

What Problem Does This Paper Address?

Birds of the crow family (corvids), such as scrub jays, are famous for their "what-where-when" memory. They can remember thousands of cache locations, the type of food hidden there (e.g., perishable crickets vs. non-perishable peanuts), and how long ago they hid it. They even seem to "plan" for the future by caching more food in places where they expect to be hungry later.

For decades, cognitive scientists have debated whether this requires episodic memory—the ability to mentally "relive" past events (mental time-travel)—or if it can be explained by simpler associative learning. The challenge is that these behaviors are so complex that they look like human-like planning. This paper addresses the gap by formalizing these competing theories into computational models to see if "simple" associative mechanisms are actually sufficient to explain the data.

Methods

The authors introduced the Plastic Caching Model, a memory-augmented reinforcement learning (RL) framework. It consists of three primary modules:

  • Motivational Control: Tracks "stomach" and "hunger" variables for different food types to simulate specific satiety (e.g., being full on peanuts but still wanting crickets).
  • Associative Memory with Systems Consolidation: A neural network that stores "what-where" associations. Crucially, it uses a multi-step consolidation process where memories move through different neural layers over time. This allows the bird to decode "when" an event happened based on which layer is currently active.
  • Plastic Caching Policy: Uses neo-Hebbian three-factor learning rules (pre-synaptic activity, post-synaptic activity, and a modulatory feedback signal) to update caching preferences based on success or failure.

To test the model, the authors created a Domain-Specific Language (DSL) to formalize 28 different behavioral experimental protocols from the literature, allowing them to run "simulated birds" through the exact same trials as real birds.

Datasets

The study performed a massive meta-analysis using:

  • 28 published behavioral experiments involving food-caching corvids (e.g., scrub jays, Eurasian jays, and magpies).
  • 1,568 data points extracted from the text and figures of these publications.
  • The data covered three main categories: Satiety experiments (hunger-driven caching), Memory experiments (what-where-when tasks), and Planning experiments (anticipating future needs).

Key Insights and Ideas

  • "When" is a spatial property of memory: Instead of a digital timestamp, the model tracks time through systems consolidation. As a memory "ages," it moves through a physical pipeline of neural layers. The bird knows how old a memory is simply by which layer "lights up" during recall.
  • Planning can be "Model-Free": What looks like "planning for the future" can be explained by updating caching weights at the moment of retrieval. If a bird finds a cache has been pilfered, it lowers its preference for that site for future caching. This "delayed reinforcement" shapes future behavior without the bird needing to explicitly "imagine" the future.
  • Sample Size Matters: The model suggests that some failed replications in bird behavioral studies (like the "breakfast planning" experiments) might simply be due to low sample sizes ($N=6$ or $8$) rather than a flaw in the theory.

Key Results

  • Sufficiency: The Plastic Caching Model (associative RL) was sufficient to reproduce the results of all 28 experiments. It performed just as well as a more complex "Planning-By-Replay" model that explicitly simulated mental time-travel.
  • Reproducibility: Using the original small sample sizes, the model reproduced the significance levels of the most relevant statistical tests in about 50% of simulations. When the number of simulated birds was increased 10-fold, reproducibility jumped to 85–90%.
  • Ablation Studies: Removing the associative memory module caused the model to fail "what-where-when" tasks, while removing motivational control caused it to fail satiety tasks, proving that both modules are necessary for the full range of corvid behavior.

Limitations and Future Work

  • Sensory Simplification: The model does not simulate actual visual input or motor movements; it assumes the bird's brain has already processed "objects" and "locations" into clean neural representations.
  • Neural Validation: While the model proposes a neural architecture, it has not yet been validated against real-time electrophysiological recordings from caching birds.
  • Discriminative Experiments: The authors suggest that future work should focus on designing new experiments specifically intended to "break" the associative model—finding the exact point where simple association fails and true mental time-travel becomes the only possible explanation.

Summary updated

Abstract

Birds of the crow family adapt food-caching strategies to anticipated needs at the time of cache recovery and rely on memory of the what, where and when of previous caching events to recover their hidden food. It is unclear if this behavior can be explained by simple associative learning or if it relies on higher cognitive processes like mental time-travel. We present a computational model and propose a neural implementation of food-caching behavior. The model has hunger variables for motivational control, reward-modulated update of retrieval and caching policies and an associative neural network for remembering caching events with a memory consolidation mechanism for flexible decoding of the age of a memory. Our methodology of formalizing experimental protocols is transferable to other domains and facilitates model evaluation and experiment design. Here, we show that memory-augmented, associative reinforcement learning without mental time-travel is sufficient to explain the results of 28 behavioral experiments with food-caching birds.
BibTeX
@inproceedings{brea2023computational,
  author = {Johanni Brea and Nicola S. Clayton and Wulfram Gerstner},
  title = {{Computational models of episodic-like memory in food-caching birds}},
  booktitle = {Nature Communications},
  year = {2023},
  doi = {10.1038/s41467-023-38570-x},
}
Figure