Abstract
Value-guided decisions are a cornerstone of cognition, yet the underlying circuit-level mechanisms remain elusive. We used reinforcement learning to train recurrent neural network models endowed with Dale's law on a battery of economic choice tasks, which revealed a two-stage computational framework. First, value estimation occurs at input level where learned weights store subjective preferences and approximate the nonlinear multiplication of reward magnitude and probability to yield expected values. This feedforward mechanism enables generalization to novel choice options. Second, option values are compared within the recurrent network, where specific connectivity patterns mediate robust winner-take-all decisions, with both excitatory and inhibitory neurons exhibiting value and choice selectivity. By training a single network on multiple tasks, we show compositional representations combining a shared computational schema with specialized neural modules. Reproducing key neurophysiological findings from the primate orbitofrontal cortex, our model unifies value computation, comparison, and generalization into a coherent framework with testable predictions.
Introduction
When choosing between a small, guaranteed reward and a larger, risky one, the brain must integrate multiple attributes, compare values, and execute a decision. While neurons in the orbitofrontal cortex (OFC) are known to track these variables, the specific circuit mechanisms have remained a "black box." Previous models often relied on non-biological shortcuts or ignored Dale’s Law—the principle that a neuron is either excitatory or inhibitory. This research develops a biologically plausible recurrent neural network (RNN) to bridge the gap between single-neuron activity and a unified theory of economic choice.
Methods and Task Design
The authors constructed an RNN of 256 neurons adhering to Dale’s Law, with a fixed 80/20 ratio of excitatory to inhibitory units. Unlike standard AI models, these neurons utilize realistic time constants and synaptic dynamics. The network was trained using Proximal Policy Optimization (PPO), a reinforcement learning algorithm that mimics how animals learn through trial and error.
The model was evaluated on a battery of five classic neuroeconomic tasks:
- Standard Task: Choosing between two goods of different quantities.
- Risky Task: Choosing between goods with varying quantities and probabilities.
- Bundles Task: Choosing between sets of multiple goods.
- Ternary Task: Choosing among three options.
- Sequential Task: Comparing options presented one after another, requiring working memory.
A Two-Stage Computational Framework
The model reveals a clear division of labor in the decision-making process:
- Valuation via Input Weights: The network approximates the product of magnitude and probability ($Value \approx Magnitude \times Probability$) using feedforward input weights. This allows the model to generalize to novel offers immediately without retraining.
- Comparison via Recurrent Inhibition: Option comparison occurs within the recurrent circuit. The authors identified a "Competitive Recurrent Inhibition" (CRI) motif, where inhibitory neurons are selectively tuned to specific choices. As one option gains dominance, its associated inhibitory neurons suppress competing representations, driving a winner-take-all decision.
Results and Neural Correspondence
The RNN achieved over 90% accuracy across all tasks, mirroring primate behavioral performance. Crucially, the internal state of the network replicated three distinct neuron types found in the primate OFC: Offer Value neurons, Chosen Value neurons, and Chosen Good neurons.
Furthermore, the study highlights the "compositional" nature of the neural code. By training on multiple tasks, the network develops a shared computational schema for valuation and comparison, recruiting specialized modules only for unique task requirements (e.g., working memory in sequential tasks). Curriculum learning—starting with simple tasks—was found to significantly accelerate this process.
Discussion and Limitations
This framework provides a testable circuit-level explanation for economic choice. However, limitations remain. The model uses simplified scalar inputs rather than complex visual scenes and focuses primarily on the OFC. Future research should explore distributed multi-regional models (including vmPFC and amygdala) and investigate how the brain might switch to alternative heuristic strategies under high uncertainty.
Summary updated