Rats exhibit similar biases in foraging and intertemporal choice tasks

Gary A. Kane, Aaron M. Bornstein, Amitai Shenhav, Robert C. Wilson, Nathaniel D. Daw, Jonathan D. Cohen

TL;DR

This paper reveals that rats exhibit consistent "myopic" decision-making biases in both foraging and intertemporal choice tasks, preferring immediate rewards over potentially larger future ones. A quasi-hyperbolic discounting model successfully explains these biases across both contexts, suggesting a common underlying mechanism for this seemingly irrational behavior.

What Problem Does This Paper Address?

Many animals, including humans, display a tendency towards "overharvesting" in foraging scenarios, meaning they stay at a food source longer than optimal, and a preference for immediate rewards over delayed ones in intertemporal choice tasks. While these two behaviors appear similar, previous attempts to explain them with a unified model of time preference have failed. This disconnect suggests a gap in our understanding of how animals make decisions under uncertainty and over time, particularly regarding the potential for a common cognitive mechanism underlying these seemingly disparate behaviors. The authors aim to bridge this gap by systematically comparing rat behavior in both foraging and intertemporal choice tasks and testing a unified computational model.

Methods

The researchers employed a combination of extensive behavioral experiments with rats and quantitative modeling.

  • Behavioral Experiments: Rats were trained on two main types of tasks:
  • Foraging Tasks: Rats had to decide whether to harvest from a current food patch or travel to a new one. Various foraging environments were manipulated, including travel time between patches, reward depletion rates, reward/delay scaling, and the placement of delays (before or after reward).
  • Intertemporal Choice Task: Rats chose between a smaller-sooner (SS) reward and a larger-later (LL) reward, with varying reward amounts and delays.
  • Quantitative Modeling: The core of the study involved fitting various computational models to the observed rat behavior. These models aimed to explain the observed biases, particularly focusing on:
  • Optimal Foraging Theory (MVT): A baseline model predicting rational behavior.
  • Subjective Costs: The idea that leaving a patch has an inherent cost.
  • Nonlinear Reward Utility: The concept that larger rewards are not perceived as proportionally more valuable.
  • Biased Time Perception: Hypotheses including underestimating post-reward delays or overestimating pre-reward delays.
  • Temporal Discounting Models: Various forms of discounting, including exponential, standard hyperbolic, constant sensitivity, and crucially, quasi-hyperbolic discounting. This model, which incorporates separate discounting rates for short- and long-term rewards, was the primary candidate for explaining a unified bias.

Model fitting and comparison were done using Bayesian Information Criterion (BIC) and expectation-maximization algorithms.

Datasets

The study utilized data generated from the behavioral experiments conducted with adult Long-Evans rats. No pre-existing datasets were used for training or evaluation. The experiments themselves generated novel datasets of rat choices and behaviors in controlled foraging and intertemporal choice paradigms.

Key Insights and Ideas

  • Unified Bias: The most significant insight is that rats exhibit remarkably similar "myopic" biases in both foraging and intertemporal choice tasks. This suggests that the tendency to favor immediate gratification isn't confined to abstract reward choices but extends to fundamental survival behaviors like foraging.
  • Quasi-Hyperbolic Discounting as a Unifying Mechanism: The quasi-hyperbolic discounting model, which accounts for a steeper discount rate for immediate rewards and a shallower rate for distant ones, proved to be the best explanation for rat behavior across both task types. This is a powerful finding, suggesting a common cognitive architecture for temporal decision-making.
  • Beyond Simple Discounting: The failure of simpler discounting models (like exponential or standard hyperbolic) to fully capture the nuances of behavior, especially in foraging, highlights the need for more sophisticated models that can account for the complex interplay of rewards, delays, and opportunity costs.
  • Sensitivity to Delay Placement: Rats showed differential sensitivity to delays depending on whether they occurred before or after reward receipt, further supporting the idea that time preferences, rather than just objective reward rates, are driving their decisions.

Key Results

  • Foraging Overharvesting: Rats consistently overharvested in foraging tasks, staying in patches longer than predicted by optimal foraging theory (MVT). This bias was influenced by factors like reward magnitude and delay placement.
  • Intertemporal Choice Bias: Rats exhibited a clear preference for smaller, sooner rewards over larger, later rewards, consistent with delay discounting.
  • Model Performance: The quasi-hyperbolic discounting model provided the best quantitative fit to rat behavior across all foraging experiments and the intertemporal choice task.
  • Cross-Task Prediction: Parameters from the quasi-hyperbolic model fit to foraging data could predict intertemporal choice behavior, and vice-versa, demonstrating the model's generality.
  • Sensitivity to Delay Type: Rats were more sensitive to pre-reward delays than post-reward delays in foraging tasks, and this sensitivity was captured by the quasi-hyperbolic model.

Limitations and Future Work

  • Model Complexity: While quasi-hyperbolic discounting was the best fit, the authors acknowledge that other models (e.g., subjective costs, nonlinear reward utility) could explain some of the observed biases, suggesting that these factors might independently contribute to suboptimal decision-making.
  • Generalizability: The study was conducted with rats. While this provides a strong animal model, future work should explore whether these findings generalize to other species, including humans.
  • Source of Time Preferences: The paper identifies that time preferences drive these biases but doesn't fully elucidate why these preferences exist or their underlying neural mechanisms. Future research could investigate the neural correlates of quasi-hyperbolic discounting in these contexts.
  • Environmental Factors: The authors suggest that future work should explore how factors like changes in the environment or specific disease states might influence these biases.
  • Core Computation: The paper posits that these biases might reflect a "core computation underlying sampling and decision-making under uncertainty and across time." Further research could directly test this hypothesis by exploring how these biases manifest in tasks involving different forms of uncertainty.

Summary updated

Abstract

Animals, including humans, consistently exhibit myopia in two different contexts: foraging, in which they harvest locally beyond what is predicted by optimal foraging theory, and intertemporal choice, in which they exhibit a preference for immediate vs. delayed rewards beyond what is predicted by rational (exponential) discounting. Despite the similarity in behavior between these two contexts, previous efforts to reconcile these observations in terms of a consistent pattern of time preferences have failed. Here, via extensive behavioral testing and quantitative modeling, we show that rats exhibit similar time preferences in both contexts: they prefer immediate vs. delayed rewards and they are sensitive to opportunity costs of delays to future decisions. Further, a quasi-hyperbolic discounting model, a form of hyperbolic discounting with separate components for short- and long-term rewards, explains individual rats' time preferences across both contexts, providing evidence for a common mechanism for myopic behavior in foraging and intertemporal choice.
BibTeX
@inproceedings{kane2019rats,
  author = {Gary A. Kane and Aaron M. Bornstein and Amitai Shenhav and Robert C. Wilson and Nathaniel D. Daw and Jonathan D. Cohen},
  title = {{Rats exhibit similar biases in foraging and intertemporal choice tasks}},
  booktitle = {eLife},
  year = {2019},
  doi = {10.7554/elife.48429},
}
Figure