Paying for Patience in Synapses, Not Spikes

When people choose between a small reward now and a larger one later, the brain has to imagine the later outcome, and that imagining is noisy. A leading account, Rationally inattentive intertemporal choice, treats discounting (the drop in how much a delayed reward is worth) as the side effect of a mind that spends limited "effort" to make its simulation of the future more precise. In that account, effort is an abstract price per bit of information. Neurophysiology suggests a more specific price. Most of the brain's energy goes to synaptic input and its cleanup, not to the spikes neurons send out. This is why blood-flow imaging and spiking can disagree, a mismatch we will call metabolic-electrophysiological decoupling. This proposal asks what happens if the precision of a forward simulation is bought with synaptic current instead of bits or spikes. The answer is a signature that bits-based and spike-based accounts do not produce: on trials that call for more precise simulation, the brain should sharpen its population code without raising average firing, and the extra cost should show up only in synaptic and metabolic measures.

The project builds a small model of a predictive state estimator, meaning a recurrent system whose internal state is a running prediction of future observations. The model uses two-compartment spiking neurons, and its cost is set on dendritic and synaptic current. It is trained on intertemporal choice tasks, and the same network is retrained with spike costs or with no cost. The cost placement that best explains existing behavior and human single-neuron data is the result. The behavioral side of the claim can be tested with a much simpler analytic model, and we use that simpler model wherever it is enough.

Core Claim

The claim is that the cost which limits how precisely an agent simulates a delayed reward is mainly synaptic. This has two consequences that separate it from existing accounts. The first is neural. When stakes or difficulty push the agent to simulate more precisely, the added precision appears as better decodability and lower trial-to-trial noise along the value direction of the population code, while mean spike rates stay flat or fall. A spike-cost model instead predicts that precision is bought with more spikes. The pure information-cost model of Gershman and Bhui says nothing about rates either way. The second consequence is behavioral. If holding a forward simulation over its internal rollout costs current at every step, the cost grows with the length of the rollout. Discounting should then approach a nonzero floor as stakes grow, rather than vanishing as precision becomes unbounded. That floor should be most visible for delays the agent actually sits through (seconds of waiting), not delays that are only described (months).

The second point needs a caveat stated up front. Nobody holds a neural state for 365 days. The "running cost" therefore applies to the length of the internal simulation, not to calendar time. Only in tasks with experienced delays does it map onto physical holding time.

Model

The network extends Energy optimization induces predictive-coding properties in a multi-compartment spiking neural network model. That paper trained two-compartment spiking neurons on MNIST, using an energy loss equal to the voltage mismatch between the apical tuft (which receives top-down prediction) and the soma (which receives bottom-up input). Predictive-coding behavior emerged, along with lower spike rates. That model is static and feedforward. We make it recurrent and predictive, borrowing the training target of predictive state representations from Predictive Representations of State and Recurrent Predictive State Policy Networks. In those methods, the hidden state is required to predict future observations, and it is initialized by two-stage regression instead of at random.

On each trial the network sees a reward magnitude $V_0$ and a delay $D$. It rolls its state forward for $n(D)$ internal steps, where $n$ increases with delay. It then emits a value estimate $\hat V$, and a softmax policy compares $\hat V$ against the immediate option. Simulation noise is added at every step, and the network can suppress it only by driving top-down apical current. The training objective is

$$\mathcal{L} = \mathbb{E}\big[\ell(\text{choice})\big] + \lambda \sum_{t=1}^{n(D)} \sum_i c_i(t),$$

Here $\ell$ is the loss from choosing the worse option, scaled by the stakes. $\lambda$ sets the energy price. The term $c_i(t)$ is the per-step cost of neuron $i$ and takes one of three forms. In the synaptic variant it is the absolute apical-somatic voltage mismatch plus total synaptic input current, motivated by Pharmacological Dissection of Intrinsic Optical Signal Reveals a Functional Coupling between Synaptic Activity and Astrocytic Volume Transient, which found that postsynaptic receptor activity accounts for more than half of the optical metabolic signal. In the spike variant it is the spike count. In the control it is zero. The architecture, data, and tuning budget are identical across the three, so only the cost placement differs.

The motivation for synaptic pricing comes from three sensory and motor findings. Mirror neurons in monkey area F5 do not adapt to the observation of repeated actions found that the local field potential adapts while spiking does not. Additive Effects of Attention and Stimulus Contrast in Primary Visual Cortex found top-down effects on subthreshold input that do not necessarily drive spikes. Fine-Scale Spatial Organization of Face and Object Selectivity in the Temporal Lobe found that imaging tracks synaptic input more closely than output. None of these studies examined valuation or delay, and that is the gap we fill.

Research Program

The program has three linked parts. First, we derive a closed-form version of the running-cost idea inside the Gershman–Bhui model, so that the behavioral predictions can be tested without any network. In this version, the cost of a given precision scales with rollout length, and the effect on discounting depends on how the per-step cost grows. Second, we train the three network variants on synthetic intertemporal choice that spans magnitudes and delays. We confirm that each reproduces hyperbolic-looking discounting and a magnitude effect. We then measure, per trial, three quantities that matter here: mean firing, value-decoding accuracy and noise along the value axis, and summed synaptic current, which serves as a stand-in for the blood-flow signal. The network is useful only if these neural signatures differ across cost placements while behavior stays matched. If behavior is matched and the signatures do not differ, the network adds nothing over the analytic model, and we will report that. Third, we test the predictions on existing human behavioral and single-neuron data.

Baselines

The naive reference point is a per-subject hyperbolic fit, $V_0/(1+kD)$, as used in Comparing Discounting of Potentially Real Rewards and Losses by Means of Functional Magnetic Resonance Imaging. It is fit with maximum likelihood and the same softmax choice rule as every other model. By construction it has no magnitude effect, so any model that cannot beat it on held-out magnitude conditions is not worth considering.

The strongest existing behavioral competitor is the full rational inattention model from Rationally inattentive intertemporal choice. The authors report that it outperformed hyperbolic models on the Chávez et al. (2017) data with more than 1,200 participants, but the file contains no log-likelihood or information-criterion values; no reported result found on Lacuna for those numbers. We will refit it ourselves on the same data, with identical cross-validation folds and the same parameter-search budget, and compare it with our running-cost variant. Beating it means better held-out likelihood specifically on the highest-stakes trials, where the two models' predictions part ways. Matching it elsewhere is acceptable.

On the neural side, the reference numbers are from Multiregional representations of intertemporal decision making in human single neurons: choice-decoding accuracy of 78% in OFC, 72.6% in amygdala, and 73.8% in hippocampus. We do not aim to beat these figures. The test is whether changes in representation strength come with changes in firing rate.

The ablation that isolates the key ingredient is the spike-cost network, plus a point-neuron version of the synaptic-cost network without separate compartments. Both get the same training budget. The synaptic-cost model counts as distinct only if it matches the others on behavior while producing a different relationship between rate and precision.

For a competence check of the predictive state module, we run it on the partially observable MuJoCo tasks used by Recurrent Predictive State Policy Networks. That paper reports that its method outperformed GRUs there, but it gives no return values; no reported result found on Lacuna for those returns. We only require our module to come within reach of an RPSP reimplementation. It does not need to beat it.

Why not just…?

Why not just add a delay-dependent cost to the Gershman–Bhui model? For behavior alone, this is the right approach, and we adopt it. The running-cost floor is derived and fit analytically, with no spiking network involved. The network is needed for a different question that the analytic model cannot pose: where the cost is paid in tissue. If the analytic variant fits behavior and the network's neural signatures do not separate the cost placements, we will drop the network.

Why not just penalize spike rates in point neurons? The biophysical evidence argues against spikes as the main energy cost, and the MNIST energy model obtained predictive behavior specifically from the compartment mismatch. Still, this is an empirical question in our setting, and the spike-cost ablation settles it. The spike-cost model predicts that hard trials, high-stakes trials, and patient subjects come with higher firing. The synaptic-cost model predicts no such rise.

Why not just use a GRU and read "energy" off its activations? A GRU has no separation between input current and output, so it cannot express decoupling at all. Any "energy" read from it is simply its activity level. It remains useful as a behavioral sanity baseline, but it cannot tell the neural hypotheses apart.

Why not just fit hyperbolic discounting? It has no mechanism for a magnitude effect, and Gershman and Bhui already showed that rational inattention fits better. It serves as the naive floor.

Proposed Experiments

The first decisive check is behavioral saturation. Using the Gershman–Bhui 221-participant data and the Chávez data, we fit discount rate as a function of magnitude and test whether it keeps falling or levels off at a positive asymptote. The pure information-cost model allows discounting to approach zero, while the running-cost model predicts a floor. The Ballard et al. (2017) justification manipulation, in which forced effort lowered discounting for small rewards, provides a second test. Our model predicts the effort benefit shrinks as delay grows, because the rollout cost increases.

The second check uses the human intracranial data. On within-unit comparisons of hard trials (value difference under $1) versus easy trials, and on slow versus fast discounters, we ask whether the stronger hippocampal representation reported by Gill et al. comes with higher mean firing. Higher decodability with flat or lower rates favors synaptic pricing, and rising rates favor spike pricing. With 9 patients, between-subject contrasts are weak and confounded by electrode placement, so the within-unit contrast is primary. If raw broadband recordings exist (the file does not say they do), extracting the local field potential would give the direct test: synaptic-band power should rise on hard trials while rates stay flat.

The third check is the experiment this work ultimately calls for. Simultaneous microwire spikes and field potentials would be recorded during a task that crosses stakes with experienced delays of a few seconds, the regime where a holding cost should matter most, as rodent waiting tasks such as Rats exhibit similar biases in foraging and intertemporal choice tasks suggest. The proposal is falsified if precision gains on high-stakes trials consistently come with spike-rate increases matched by field-potential changes, because that would mean the brain pays for patience in spikes after all.