Evidence for hierarchically-structured reinforcement learning in humans.

Maria K. Eckstein, Anne CollinseScholarship 2018

Abstract

Flexibly adapting behavior to different contexts is a critical\ncomponent of human intelligence. It requires knowledge to\nbe structured as coherent, context-dependent action rules, or\ntask-sets (TS). Nevertheless, inferring optimal TS is compu-\ntationally complex. This paper tests the key predictions of a\nneurally-inspired model that employs hierarchically-structured\nreinforcement learning (RL) to approximate optimal inference.\nThe model proposes that RL acts at two levels of abstrac-\ntion: a high-level RL process learns context-TS values, which\nguide TS selection based on context; a low-level process learns\nstimulus-actions values within TS, which guide action selec-\ntion in response to stimuli. In our novel task paradigm, we\nfound evidence that participants indeed learned values at both\nlevels: not only stimulus-action values, but also context-TS\nvalues affected learning and TS reactivation, and TS values\nalone determined TS generalization. This supports the claim\nof two RL processes, and their importance in structuring our\ninteractions with the world.
BibTeX
@inproceedings{eckstein2018evidence,
  author = {Maria K. Eckstein and Anne Collins},
  title = {{Evidence for hierarchically-structured reinforcement learning in humans.}},
  booktitle = {eScholarship},
  year = {2018},
}

Loading related content...

Figure