Distributional coding of associative learning within projection-defined populations of midbrain dopamine neurons

Riccardo Avvisati, Anna-Kristin Kaufmann, Callum J. Young, Gabriella E. Portlock, Sophie Cancemi, Rui Ponte Costa, Peter J. Magill, Paul D. DodsonbioRxiv 2022 · DOI

TL;DR

Midbrain dopamine neurons, long thought to send a uniform "teaching signal" for learning, actually exhibit diverse firing patterns. This study reveals that this heterogeneity isn't random but organized by which part of the striatum the neurons project to, suggesting specialized roles for dopamine signaling in different aspects of learning and behavior.

What Problem Does This Paper Address?

For decades, the prevailing view was that midbrain dopamine neurons broadcast a single, unified signal to the striatum, representing the "reward prediction error" – the difference between expected and actual rewards. This signal was believed to be the primary driver of associative learning. However, recent evidence has pointed to significant diversity in dopamine neuron activity and signaling. The crucial unanswered question was: how is this heterogeneity organized, and how does it serve the distinct functions of the various striatal regions that dopamine neurons innervate? This paper aims to unravel this organizational logic, moving beyond a uniform signal model to understand how projection-specific dopamine signaling supports specialized behavioral roles.

Methods

The researchers employed a multi-pronged approach:

  • Behavioral Training: Mice were trained in a Pavlovian conditioning paradigm where an auditory cue predicted reward delivery. This allowed them to observe anticipatory licking, a learned behavior.
  • Electrophysiology: Individual dopamine neurons were recorded extracellularly while the mice performed the learned task.
  • Molecular Identification: Neurons were juxtacellularly labeled with neurobiotin to precisely determine their location and then stained for tyrosine hydroxylase (TH) to confirm their dopaminergic identity.
  • Projection Targeting: To identify which striatal region a neuron projected to, retrograde tracer (cholera toxin B subunit - CTB) was injected into different striatal areas (DMS, DLS, VLS, NAc core). This allowed researchers to correlate neuron location and protein expression with their projection targets.
  • Protein Expression Analysis: Immunohistochemistry was used to examine the expression of specific proteins (Aldh1a1, Sox6, calbindin) in dopamine neurons. Combinations of these markers, along with anatomical location, were used to classify projection-defined populations.
  • Computational Modeling: A temporal difference (TD) learning model was used to simulate how different "distributional coding" strategies (approximating the observed firing patterns) might impact learning accuracy in state-value associations and motivational control tasks.

Datasets

Not explicitly discussed as distinct datasets for training/evaluation in the context of machine learning. The study generated its own electrophysiological and immunohistochemical data from experimental animals.

Key Insights and Ideas

  • Beyond Uniformity: The core takeaway is that dopamine signaling is not a monolithic message. The heterogeneity observed is not random but systematically organized by projection target.
  • Projection-Specific Coding: Different striatal regions receive distinct dopamine signals. This suggests a "distributional coding" where the pattern of dopamine neuron activity, rather than just the average, carries specific information.
  • Functional Specialization: This projection-specific coding appears to tailor dopamine signals to support the specialized roles of different striatal regions, such as state-value learning (medial striatum) versus effortful action and motivation (lateral striatum).
  • Multiplexing is Common: Individual dopamine neurons often encode multiple task variables simultaneously (e.g., cue, reward, licking, movement), a phenomenon known as multiplexing.
  • Anatomical Subdivisions Aren't Enough: Simply dividing dopamine neurons into VTA and SNc populations didn't fully explain the observed heterogeneity in firing patterns. Projection target proved to be a more powerful organizing principle.
  • "Hot Spots" for Projections: While encoding wasn't strictly focal, there appear to be "hot spots" within the midbrain (e.g., medial SNc, VTA) where neurons projecting to the same striatal region are clustered.

Key Results

  • Heterogeneous Encoding: Dopamine neurons showed diverse responses to cues and rewards, with many encoding variables beyond just reward prediction error, including licking and movement.
  • Projection-Defined Populations: By combining protein expression and location, four distinct dopamine neuron populations were identified, each projecting to a specific striatal region (DMS, DLS, VLS, NAc core).
  • Distinct Firing Patterns: These projection-defined populations exhibited different firing patterns. For instance, DMS-projecting neurons showed a decrease in firing at cue onset and no increase at reward, while VLS-projecting neurons increased firing after reward. NAc core-projecting neurons showed delayed increases in firing, coinciding with licking.
  • Computational Advantage: TD modeling predicted that projection-specific distributional coding could maximize dynamic range and tailor signals. Agents with "narrow" distributions (like DMS/NAc core) were better at state-value learning, while those with "broader" or "skewed" distributions (like VLS/DLS) were better at motivational control and evaluating larger rewards.

Limitations and Future Work

  • Causality: While the study demonstrates correlations between projection targets, protein expression, and firing patterns, future work could directly test the causal role of these projection-defined populations in behavior using optogenetics or chemogenetics.
  • Cue Volume: The authors suggest that the modest volume of the auditory cue used might explain the lack of a strong population-level response to the cue, and that louder cues or omitted rewards might reveal different cue-related signaling.
  • Marker Combinations: For DMS and VLS projections, the marker expression was less binary, suggesting potential for further subdivisions or more complex molecular identities within these populations.
  • Disambiguating Signals: The precise meaning of licking-related signals and their potential overlap with motor responses needs further investigation.
  • Broader Reward Spectrum: While the study explored familiar and motivating rewards, future work could investigate how these projection-defined populations respond across an even wider spectrum of reward values and contingencies.

Summary updated

Abstract

ABSTRACT Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. While this teaching signal is often considered to be uniform, recent evidence instead supports diversity in dopamine signaling. However, it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. We defined combinations of protein expression and cellular localization to sort recorded neurons according to the striatal regions they innervate. Our findings show that reward information and task variables are not only heterogeneously encoded, with multiplexing, but also differentially distributed across populations of dopamine neurons projecting to different regions of striatum. These data, supported by computational modelling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate the specialized roles of different striatal regions.
BibTeX
@misc{avvisati2022distributional,
  author = {Riccardo Avvisati and Anna-Kristin Kaufmann and Callum J. Young and Gabriella E. Portlock and Sophie Cancemi and Rui Ponte Costa and Peter J. Magill and Paul D. Dodson},
  title = {{Distributional coding of associative learning within projection-defined populations of midbrain dopamine neurons}},
  year = {2022},
  doi = {10.1101/2022.07.18.500429},
  howpublished = {bioRxiv},
}
Figure