Temporal chunking as a mechanism for unsupervised learning of task-sets

Flora Bouchacourt, Stefano Palminteri, Étienne Koechlin, Srdjan Ostojic

TL;DR

This paper proposes that humans learn complex task-sets through an unsupervised "chunking" mechanism, where temporally close stimulus-response associations are grouped together. A novel network model demonstrates this, showing that learning is more effective when chunking happens on a slower timescale than individual association learning. This mechanism explains behavioral variability and is supported by neural data from prefrontal cortex.

What Problem Does This Paper Address?

Humans are remarkably adept at switching between different sets of rules or "task-sets" depending on the situation. For example, in the Stroop task, the same visual stimulus can require different responses based on whether you're reading the word or identifying its color. While we know people can learn and use these task-sets, the underlying neural and computational mechanisms for this learning process remain largely unknown. Existing models often describe behavior abstractly but lack biological grounding. This paper aims to bridge this gap by proposing and testing a specific, biologically plausible mechanism for task-set learning: unsupervised temporal chunking.

Methods

The authors developed a novel computational model consisting of two interacting networks:

  • Associative Network (AN): This network learns individual stimulus-response associations using a reward-modulated Hebbian plasticity rule. It's a simplified model of a decision-making circuit.
  • Task-Set Network (TN): This network is designed to learn "chunks" of stimulus-response associations that occur in temporal proximity. It uses unsupervised, temporal Hebbian learning. When associations are frequently paired, their corresponding neurons in the TN become linked (chunked). This chunking then generates an "inference signal" that biases the AN towards the correct associations for the current task-set.

A key aspect of the model is the timescale of learning: the TN's learning rate (chunking) must be slower than the AN's learning rate (individual associations) for effective task-set learning. The model was fitted to behavioral data from human subjects on a subject-by-subject basis. Additionally, fMRI data was used to identify neural correlates of the model's inference signal.

Datasets

The study utilized data from two previously published experiments:

  • Experiment 1 (Collins and Koechlin, 2012): Behavioral data from 22 human subjects performing a task-set learning experiment.
  • Experiment 2 (Donoso et al., 2014): Behavioral and fMRI data from 40 human subjects performing a similar task-set learning experiment.

The authors did not introduce new datasets but rather re-analyzed existing ones to test their model's predictions.

Key Insights and Ideas

  • Temporal Proximity as the Basis for Task-Set Learning: The core idea is that associations occurring close in time are automatically grouped, forming "chunks" that represent task-sets. This is a form of unsupervised learning, requiring no explicit instruction.
  • The Importance of Learning Timescales: The model's success hinges on the TN learning associations on a slower timescale than the AN. This prevents the TN from being overwhelmed by noisy or transient associations, allowing it to capture more stable, rule-based patterns.
  • Task-Set Retrieval as a "Shortcut": Once a task-set is chunked, retrieving even a single association within that set can trigger the inference signal, allowing the AN to quickly "guess" the correct responses for other associations in the same task-set. This explains rapid performance improvements after the first correct trial in an episode.
  • Adaptive and Maladaptive Retrieval: The inference signal can be beneficial, leading to faster and more accurate performance. However, it can also be maladaptive if the chunked associations are incorrect (e.g., due to misleading feedback), leading to errors.
  • Neural Correlates in Prefrontal Cortex: The study identified specific regions in the dorsomedial and dorsolateral prefrontal cortex as being involved in task-set retrieval, with stronger engagement when the retrieval was used to improve performance.

Key Results

  • Behavioral Validation: The model, when fitted to individual subjects, accurately predicted behavioral patterns, including the rapid performance increase after the first correct trial in a recurrent session (where task-sets repeat).
  • Inter-Subject Variability Explained: The model's "inference strength" parameter, which quantifies how strongly the TN influences the AN, was found to explain individual differences in task-set learning ability. Subjects classified as "exploiting" task-set structure showed higher inference strength.
  • Reaction Time Support: Chunked trials, predicted by the model to involve task-set retrieval, were associated with significantly lower reaction times, further supporting the idea of a cognitive shortcut.
  • Neural Correlates: BOLD signal in dorsomedial and dorsolateral prefrontal cortex positively correlated with the model's task-set inference signal, particularly in the recurrent session where task-set retrieval was crucial.

Limitations and Future Work

  • Model Simplification: The authors acknowledge that their model is a simplified representation. For instance, the inference signal is implemented directly by modulating synaptic weights, whereas biological implementations might be more complex.
  • Biological Plausibility of Gating: While the underlying mechanisms (Hebbian learning, mixed-selectivity) are biologically plausible, the precise biological instantiation of the "gating" mechanism for task-set retrieval remains an open question. Future work could explore more complex neural architectures.
  • Uncertainty Estimation: The paper suggests that future work could investigate how an estimate of uncertainty (or reliability over task-sets) is retrieved from the synaptic weights of their model.
  • Overlapping Task-Sets: The model makes predictions for scenarios with overlapping task-sets, suggesting performance decrements when associations are shared, which could be tested experimentally.

Summary updated

Abstract

Depending on environmental demands, humans can learn and exploit multiple concurrent sets of stimulus-response associations. Mechanisms underlying the learning of such task-sets remain unknown. Here we investigate the hypothesis that task-set learning relies on unsupervised chunking of stimulus-response associations that occur in temporal proximity. We examine behavioral and neural data from a task-set learning experiment using a network model. We first show that task-set learning can be achieved provided the timescale of chunking is slower than the timescale of stimulus-response learning. Fitting the model to behavioral data on a subject-by-subject basis confirmed this expectation and led to specific predictions linking chunking and task-set retrieval that were borne out by behavioral performance and reaction times. Comparing the model activity with BOLD signal allowed us to identify neural correlates of task-set retrieval in a functional network involving ventral and dorsal prefrontal cortex, with the dorsal system preferentially engaged when retrievals are used to improve performance.
BibTeX
@inproceedings{bouchacourt2020temporal,
  author = {Flora Bouchacourt and Stefano Palminteri and Étienne Koechlin and Srdjan Ostojic},
  title = {{Temporal chunking as a mechanism for unsupervised learning of task-sets}},
  booktitle = {eLife},
  year = {2020},
  doi = {10.7554/elife.50469},
}
Figure