Dopamine dynamics in nucleus accumbens across reward-based learning of goal-directed whisker-to-lick sensorimotor transformations in mice

Jun Huang, Sylvain Crochet, Carmen Sandi, Carl C.h. PetersenHeliyon 2024 · DOI

TL;DR

This study uses advanced fiber photometry to track dopamine release in mice as they learn a whisker-guided task. They found that dopamine signals in the nucleus accumbens not only signal reward but also develop in response to sensory cues that predict reward, suggesting a role in learning and sensorimotor transformation.

What Problem Does This Paper Address?

Understanding how the brain learns and adapts motor behaviors based on rewards is a fundamental question in neuroscience. While dopamine is known to be crucial for reward-based learning, the precise dynamics of dopamine release in response to sensory cues, motor actions, and actual rewards, especially during the learning process, are not fully understood. Specifically, how these dopamine signals evolve as an animal masters a new sensorimotor skill remains an open area. This paper aims to disentangle these different aspects of dopamine signaling during the acquisition of a goal-directed whisker-to-lick task in mice.

Methods

The researchers employed longitudinal fiber photometry to measure dopamine dynamics in the nucleus accumbens (NAc) of head-restrained mice. They used a genetically encoded fluorescent dopamine sensor, dLight1.1, expressed in NAc neurons via adeno-associated virus (AAV) injection. This allowed for high temporal resolution monitoring of dopamine release.

Key techniques include:

  • Fiber photometry: Measuring fluorescence changes in real-time through an implanted fiber optic cannula.
  • Genetically encoded fluorescent sensors (dLight1.1): A specific tool for detecting dopamine.
  • Behavioral paradigms: A series of carefully designed tasks (Free licking, Whisker detection, Delayed reward, Reward omission) to isolate different components of the learning process (reward acquisition, sensory cue detection, motor action, reward prediction).
  • Head-restrained setup: Allows for precise control of sensory stimuli and behavioral measurements.
  • Piezo film sensor: Used to accurately detect licking events.

Datasets

Not discussed in the paper. The study uses data collected from their own experimental setup with mice.

Key Insights and Ideas

  • Dopamine signals reward, but also prediction: The NAc shows a clear dopamine surge upon receiving a water reward. Crucially, as mice learned to associate a whisker deflection with reward, a dopamine signal also emerged in response to the whisker stimulus before reward delivery. This suggests dopamine isn't just about the reward itself, but also about anticipating it.
  • Learning sculpts dopamine responses: The sensory-evoked dopamine signal wasn't present initially but developed over days and even within single learning sessions, mirroring the improvement in task performance. This highlights dopamine's dynamic role in skill acquisition.
  • Satiation matters: The dopamine response to reward diminished as mice became sated within a session, indicating that the motivational state strongly influences dopamine signaling.
  • Unrewarded effort has a distinct signature: Unrewarded licking, especially when reward was unexpectedly omitted, was associated with reduced dopamine fluorescence, potentially signaling a negative prediction error or the cost of effort without reward.

Key Results

  • Reward-evoked dopamine: Rewarded licking in the "Free licking" task elicited a significant increase in NAc dopamine (z-score ~1.63), while unrewarded licking led to a decrease (z-score ~-0.79).
  • Sensory cue learning: In the "Whisker detection" task, a sensory-evoked dopamine signal developed over three days of training. The early response to the whisker stimulus in "Hit" trials (stimulus + lick + reward) increased significantly from Day 1 to Day 3.
  • Within-session learning: The development of the sensory-evoked dopamine signal was also observed within individual learning sessions, correlating with improved lick probability.
  • Water devaluation: The dopamine reward signal decreased significantly as mice became sated within a session, with rewarded licking responses dropping from ~4.87 to ~0.93 in the Free licking task over the course of a session.

Limitations and Future Work

  • Limitations:
    • The study primarily used male mice, and sex differences in dopamine signaling are known.
  • While the dLight sensor is powerful, it measures bulk dopamine release in a region, not single-neuron activity or specific dopamine receptor subtypes.
  • The interpretation of dopamine signals is correlational; direct causal manipulation experiments are needed.
  • Future Work:
    • Investigate sex differences in dopamine signaling during learning.
  • Combine fiber photometry with other techniques (e.g., electrophysiology with Neuropixels, multi-color photometry) to measure dopamine alongside neuronal activity or other neurotransmitters.
  • Employ optogenetic or chemogenetic tools to directly manipulate dopamine signaling and establish causal links to learning and behavior.
  • Explore the specific neuronal circuits upstream of the NAc that drive these learning-induced dopamine responses.
  • Further investigate the role of dopamine in motor initiation and its potential contribution to the timing of licking.

Summary updated

Abstract

The synaptic and neuronal circuit mechanisms underlying reward-based learning remain to be fully determined. In the mammalian brain, dopamine release in nucleus accumbens is thought to contribute importantly to reward signals for learning and promoting synaptic plasticity. Here, through longitudinal fiber photometry of a genetically-encoded fluorescent sensor, we investigated dopamine signals in the nucleus accumbens of thirsty head-restrained mice as they learned to lick a liquid reward spout in response to single deflections of the C2 whisker across varying reward contingencies. Reward delivery triggered by well-timed licking drove fast transient dopamine increases in nucleus accumbens. On the other hand, unrewarded licking was overall associated with reduced dopamine sensor fluorescence, especially in trials where reward was unexpectedly omitted. The dopamine reward signal upon liquid delivery decreased within individual sessions as mice became sated. As mice learned to lick the reward spout in response to whisker deflection, a fast transient sensory-evoked dopamine signal developed, correlating with the learning of the whisker detection task across consecutive training days, as well as within single learning sessions. The well-defined behavioral paradigm involving a unitary stimulus of a single whisker as a reward-predicting cue along with precisely quantified licking allows untangling of sensory, motor and reward-related dopamine signals and how they evolve across the learning of whisker-dependent goal-directed sensorimotor transformations. Our longitudinal measurements of dopamine dynamics across reward-based learning are overall consistent with the hypothesis that dopamine could play an important role as a reward signal for reinforcement learning.
BibTeX
@inproceedings{huang2024dopamine,
  author = {Jun Huang and Sylvain Crochet and Carmen Sandi and Carl C.H. Petersen},
  title = {{Dopamine dynamics in nucleus accumbens across reward-based learning of goal-directed whisker-to-lick sensorimotor transformations in mice}},
  booktitle = {Heliyon},
  year = {2024},
  doi = {10.1016/j.heliyon.2024.e37831},
}
Figure