Rapid Sensorimotor Reinforcement in the Olfactory Striatum

Daniel Millman, Venkatesh N. MurthybioRxiv 2019 · DOI

TL;DR

This study reveals that the olfactory tubercle (OT), a part of the olfactory striatum, rapidly develops an explicit representation of reward value for odors within minutes of learning. This contrasts with the posterior piriform cortex (pPC), which shows no such explicit reward coding even after extensive training, suggesting the OT is a key player in rapid odor-reward learning.

What Problem Does This Paper Address?

Understanding how the brain learns to associate sensory cues with rewards is fundamental to neuroscience. In olfaction, while it's known that rodents can quickly learn to link specific odors with rewards, the neural circuits responsible for this rapid sensorimotor reinforcement remain largely unknown. Researchers have identified the olfactory tubercle (OT) and posterior piriform cortex (pPC) as potential candidates for processing this information, but their specific roles and the timing of reward representation within these areas during learning were unclear. This paper aims to pinpoint where and how reward information is encoded in the olfactory system during the process of learning odor-reward associations.

Methods

The researchers trained head-restrained mice in a novel odor-reward categorization task. They used a custom-built 16-odor olfactometer to deliver stimuli and recorded neural activity from single neurons in both the pPC and OT using multi-tetrode drives. Respiration was monitored to precisely align neural activity with sniffing behavior. To analyze neural responses, they employed:

  • Peri-stimulus time histograms (PSTHs): To visualize neuronal firing patterns relative to odor onset and sniffing.
  • Area Under the Receiver-Operating Characteristic Curve (auROC): To quantify the degree of odor and reward selectivity in individual neurons.
  • Support Vector Machines (SVMs): For decoding analyses to determine if reward valence could be predicted from population-level neural activity.

A key aspect of their approach was to study both "familiar odors" (learned in prior sessions) and "novel odors" (introduced within the same recording session) to track the dynamics of learning.

Datasets

The study utilized data collected from experimentally trained mice. No pre-existing public datasets were used. The experimental setup involved a panel of eight odors, with four being rewarded and four unrewarded. This panel was systematically manipulated by replacing familiar odors with novel ones to study the learning process.

Key Insights and Ideas

  • Rapid Reward Encoding in OT: The most striking finding is the swift emergence of reward selectivity in the OT, occurring within the first sniff (50-100ms) of a novel odor. This suggests the OT is primed to quickly integrate reward information.
  • pPC Lacks Explicit Reward Code: In stark contrast, the pPC showed no clear, explicit representation of reward valence, even after prolonged training. This implies that while pPC might process odor identity, it's not the primary site for rapid reward value assignment.
  • Striatal vs. Cortical Model: The results strongly support a model where the olfactory striatum (specifically the OT) plays a more central role in rapid odor-sensorimotor learning than the olfactory cortex (pPC).
  • Multiplexing of Odor and Reward: OT neurons appear to encode both odor identity and reward valence simultaneously, allowing for efficient processing and flexible learning.
  • Behavioral Relevance: The degree of reward coding in OT was correlated with the mouse's behavioral performance, highlighting the functional significance of this neural representation.

Key Results

  • OT Reward Selectivity: Reward selectivity in OT neurons emerged within the first 300ms of odor sampling, preceding motor actions like licking. This was observed in 46% of OT cells.
  • pPC Reward Selectivity: Reward selectivity in pPC was largely absent, with significant deviations scattered throughout the trial and not aligned with task events.
  • Decoding Performance: SVM classifiers could reliably decode reward valence from OT activity using as few as 20 cells, achieving performance comparable to well-trained mice. Decoding from pPC data was significantly poorer and declined rapidly with an increasing number of odors.
  • Novel Odor Learning: Mice learned novel odor-reward associations rapidly, achieving high accuracy within the first session. This learning was mirrored by changes in OT neural activity, with reward selectivity developing in parallel with behavioral performance.

Limitations and Future Work

  • Odor Specificity: While the study used a panel of odors, the extent to which these findings generalize to all possible odors and reward contingencies remains to be fully explored.
  • Single-Neuron vs. Population: While single-neuron reward selectivity was weak in pPC, the possibility of distributed or less explicit coding mechanisms in pPC cannot be entirely ruled out without further investigation.
  • Causality: The study demonstrates correlation between OT activity and reward learning. Future work could employ optogenetic or chemogenetic manipulations to establish causal links between OT activity and reward-guided behavior.
  • Circuitry: Further research could delve deeper into the specific microcircuits within the OT that mediate this rapid reward learning and how they interact with other brain regions involved in decision-making and motor control.

Summary updated

Abstract

Abstract Rodents can successfully learn multiple, novel stimulus-response associations after only a few repetitions when the contingencies predict reward. The circuits modified during such reinforcement learning to support decision making are not known, but the olfactory tubercle (OT) and posterior piriform cortex (pPC) are candidates for decoding reward category from olfactory sensory input and relaying this information to cognitive and motor areas. Here, we show that an explicit representation for reward category emerges in the OT within minutes of learning a novel odor-reward association, whereas the pPC lacks an explicit representation even after weeks of overtraining. The explicit reward category representation in OT is visible in the first sniff (50-100ms) of an odor on each trial, and precedes the motor action. Together, these results suggest that coding of stimulus information required for reward prediction does not occur within olfactory cortex, but rather in circuits involving the olfactory striatum.
BibTeX
@misc{millman2019rapid,
  author = {Daniel Millman and Venkatesh N. Murthy},
  title = {{Rapid Sensorimotor Reinforcement in the Olfactory Striatum}},
  year = {2019},
  doi = {10.1101/730697},
  howpublished = {bioRxiv},
}
Figure