People and animals treat delayed rewards as worth less than immediate ones, and the usual explanation is a fixed taste for the present. A competing account holds that discounting is a rational correction for imprecision: we evaluate a future reward by mentally simulating it, those simulations get noisier the further ahead they reach, and a sensible estimator shrinks noisy estimates toward a default of "probably nothing." This proposal makes that account concrete. It identifies the noisy simulator with an internal forward model, the learned predictor that brains and robots use to forecast the next state from the current state and a copy of the outgoing action. It then claims that how steeply someone discounts a delay is set by how fast their own forward model's multi-step predictions degrade over that delay, rather than by clock time.
The claim has a sharp, testable consequence. Two delays of identical length, ending in identical guaranteed rewards, should be discounted differently when the process filling the delay is one the person can predict well versus poorly. The difference should hold even when the objective risk is the same and only the person's training on that process differs. It should also be predictable in advance from an independent measurement of that person's prediction error, with no extra fitted parameters.
Core Claim
Rationally inattentive intertemporal choice supplies the inference scheme we reuse. That paper models valuation as Bayesian shrinkage of a noisy simulated reward toward a zero prior, with mental effort buying precision, and it explains the magnitude effect and the link between stakes and choice randomness. Under its scheme, a reward of size $A$ at delay $D$ is valued at
$$V(A,D) = A \cdot \frac{\sigma_A^2}{\sigma_A^2 + \sigma_\epsilon^2(D)},$$
where $\sigma_A^2$ is the prior variance over reward sizes and $\sigma_\epsilon^2(D)$ is the variance of the simulation noise at delay $D$. If $\sigma_\epsilon^2(D)$ grows linearly in $D$, this reduces to the hyperbolic form $A/(1+kD)$ with $k$ equal to the noise growth rate divided by $\sigma_A^2$. That paper simply assumes the noise grows with delay. Our proposal replaces the assumption with a measurable quantity:
$$\sigma_\epsilon^2(D) \approx \mathbb{E}\,\big\| s_{t+D} - \hat{s}_{t+D} \big\|^2,$$
the expected squared error of the agent's own forward model when it is rolled out $D$ steps through the dynamics that lie between choice and reward. Here $s_{t+D}$ is the true state at the end of the delay and $\hat{s}_{t+D}$ is the forward model's prediction of it.
Hyperbolic discounting then becomes a special case, the one where rollout error accumulates linearly, much like a random walk. If error saturates, as it does in highly predictable dynamics, discounting should flatten at long delays. If error compounds, as it does in poorly learned or unstable dynamics, discounting should steepen beyond hyperbolic.
This addresses the "abstraction disconnect" noted in the research file, namely that rational inattention leaves simulation noise ungrounded. It also takes up the open problem raised by Predictive learning: its key role in early cognitive development, which builds forward models that learn by minimizing prediction error and shows they support self-other distinction and goal-directed action. That work leaves open how such models reach long-horizon deliberation. We test one specific route: forward-model fidelity feeding valuation.
Research Program
The work has three parts.
The task. We build an interactive intertemporal choice task in which the delay is a sequence of steps through a low-dimensional controlled dynamical system. One example is a cursor driven by the participant's inputs under a hidden linear-plus-noise transition rule whose predictability we set. Participants first learn the dynamics through closed-loop interaction, in the spirit of A computational model of infant sensorimotor exploration in the mobile paradigm. They then choose between smaller-sooner and larger-later options. The later reward is delivered after $D$ steps of that system and is explicitly guaranteed regardless of where the system ends up.
The independent measurement. Interleaved probe trials ask participants to predict the system state $n$ steps ahead after a given action sequence. This yields a per-participant, per-condition curve of rollout error against horizon. That curve, plugged into the shrinkage equation with $\sigma_A^2$ fixed from a separate baseline block, produces a parameter-free prediction of the discount curve in that condition.
The computational agent. In parallel, we train an artificial agent's neural forward model on the same environments with varying amounts of data, following the efference-copy architecture of Using efference copy and a forward internal model for adaptive biped walking. That paper trained a three-layer network on motor commands and joint angles to predict accelerations, reaching a mean squared error of 0.0011 and raising slope-walking success from 0% to 90%. We use the agent's measured rollout error as $\sigma_\epsilon^2(D)$ and ask whether its emergent discounting matches the human pattern across conditions.
The decisive manipulation crosses environment predictability with training dose. The training-dose factor matters most. Holding the environment fixed and varying only how much experience a participant gets changes their forward-model error while leaving every objective property of the delay unchanged, including delivery probability and the variance of the true dynamics.
Baselines
The naive reference point is a single hyperbolic discount rate $k$ per participant, shared across all conditions. This is the standard behavioral-economics model, as used in Comparing Discounting of Potentially Real Rewards and Losses by Means of Functional Magnetic Resonance Imaging. It predicts no condition effect and is the floor any account must beat.
The strongest existing competitor is the full rational inattention model of Rationally inattentive intertemporal choice. In it, noise grows with clock delay and is modulated by stake-dependent effort. That paper reports it outperforms hyperbolic discounting on the Chávez et al. (2017) re-analysis of more than 1,200 participants, and on the gershman20 data with 221 participants. However, no reported numerical fit metric for either dataset was found on Lacuna, so we will refit it ourselves.
We give this model the same choice data, the same likelihood, and the same per-participant fitting procedure as ours, including its effort parameter. The only difference is that its noise depends on clock steps while ours comes from the measured rollout error. Beating it means better held-out log-likelihood on choices from conditions not used in fitting, and in particular capturing the training-dose effect that its clock-time noise cannot represent.
We also fit two further baselines with the same tuning budget. The first is the quasi-hyperbolic present-bias model, which was the best-fitting account across foraging and intertemporal choice in Rats exhibit similar biases in foraging and intertemporal choice tasks. The second is a hierarchical model-free reinforcement-learning agent with per-level adjusted discount factors, following Discounting and Drug Seeking in Biological Hierarchical Reinforcement Learning. That model has no forward simulator and values options from cached estimates.
The upper reference is a descriptive model with a separate free $k$ for every condition. Ours should come close to its fit while using zero condition-specific parameters.
The key ablation isolates the forward model. We feed the shrinkage equation the objective variance of the environment instead of the participant's measured prediction error. If this ablation does as well as the full model, the internal model adds nothing beyond environmental statistics.
Following Modelling dataset bias in machine-learned theories of economic decision-making, every model carries the same random-guessing mixture component. That paper found that modeling 27.6% of online participants as random guessers was needed for a laboratory model to fit online data, so decision noise must not masquerade as discounting.
Why not just…?
Why not just fit a separate hyperbolic $k$ in each condition? That will always fit at least as well, because it has the most parameters. But it explains nothing and predicts nothing for a new environment or a new training dose. Our test is predictive: condition-specific discounting must be forecast from the independently measured rollout-error curve before choices are seen. If the parameter-free prediction falls far short of the per-condition fit, the claim fails.
Why not just use rational inattention with noise proportional to delay? This is the real threat. It already explains the magnitude effect and stake-dependent randomness, and on standard text-based tasks such as gershman20 our model should reduce to it, a check we run explicitly. The two accounts come apart only where clock delay and predictability dissociate. The training-dose manipulation is built exactly for this: equal clock delays, equal objective dynamics, unequal internal-model error. If discounting does not move with training dose, clock-time rational inattention wins and we will say so.
Why not just call it risk? Unpredictable dynamics might make participants doubt that the reward will arrive. That would be risk aversion, not simulation noise. We address this three ways. Delivery is guaranteed and explained. Participants report their subjective probability of delivery on every block. Most importantly, the training-dose contrast keeps objective risk identical across arms. Any residual effect of reported doubt is entered as a covariate, and the forward-model effect must survive it.
Why not just subjective time compression? Sense of agency based on action and causation: The mechanism of intentional binding effect for voluntary action reviews how actions whose consequences the forward model predicts well feel closer in time to those consequences. That review covers windows of about 200 ms for the motor mechanism and up to 4,000 ms for the causal one. Better-predicted delays might therefore simply feel shorter, which would lower discounting through perceived $D$ rather than through noise. We collect duration reproduction judgments for each delay and test whether rollout error predicts discounting after perceived duration is controlled for. If perceived duration fully absorbs the effect, the mechanism is time perception, not shrinkage. That is still a forward-model result, but a different one.
Proposed Experiments
The first experiment is the human factorial study described above. It crosses two environment predictability levels with two training doses and uses four delay lengths, so that the shape of the discount curve can be distinguished, not just its level. This also addresses the single-delay limitation of earlier imaging designs. Three checks are decisive. First, discounting should be steeper under low training dose in the same environment. Second, the individual rollout-error curve should predict the individual discount curve's shape, including flattening when error saturates, better than clock-time rational inattention on held-out conditions. Third, the effect should survive controls for reported delivery probability and perceived duration.
The second experiment is the matched artificial agent. Its forward model is trained on increasing amounts of data, and the question is whether its rollout error, passed through the same shrinkage rule, reproduces the human condition effects. This tells us whether the mechanism is sufficient without any human-specific assumptions.
The third experiment is a consistency check on existing benchmarks. On gershman20 and the Chávez et al. re-analysis, where delays are described in text rather than experienced, the model collapses to linear noise growth. It should match rational inattention there rather than lose to it.
A positive result would turn a fitted impulsivity parameter into a consequence of how well an agent models its own world. It would also give a concrete hypothesis for the tension in the file between prediction deficits, such as those reported in Impaired Speaking-Induced Suppression in Alzheimer’s Disease, and valuation deficits as sources of impulsive choice. That hypothesis is left for follow-up work.
References
- Rationally inattentive intertemporal choice
- Predictive learning: its key role in early cognitive development
- A computational model of infant sensorimotor exploration in the mobile paradigm
- Using efference copy and a forward internal model for adaptive biped walking
- Comparing Discounting of Potentially Real Rewards and Losses by Means of Functional Magnetic Resonance Imaging
- Rats exhibit similar biases in foraging and intertemporal choice tasks
- Discounting and Drug Seeking in Biological Hierarchical Reinforcement Learning
- Modelling dataset bias in machine-learned theories of economic decision-making
- Sense of agency based on action and causation: The mechanism of intentional binding effect for voluntary action
- Impaired Speaking-Induced Suppression in Alzheimer’s Disease