Abstract
How does time pressure influence exploration and decision-making? We investigated this question with several four-armed bandit tasks manipulating (within subjects) expected reward, uncertainty, and time pressure (limited vs. unlimited). With limited time, people have less opportunity to perform costly computations, thus shifting the cost-benefit balance of different exploration strategies. Through behavioral, reinforcement learning (RL), reaction time (RT), and evidence accumulation analyses, we show that time pressure changes how people explore and respond to uncertainty. Specifically, participants reduced their uncertainty-directed exploration under time pressure, were less value-directed, and repeated choices more often. Since our analyses relate uncertainty to slower responses and dampened evidence accumulation (i.e., drift rates), this demonstrates a resource-rational shift towards simpler, lower-cost strategies under time pressure. These results shed light on how people adapt their exploration and decision-making strategies to externally imposed cognitive constraints.
Introduction
In any learning environment, we face the exploration-exploitation dilemma: do you stick with the "tried and true" (exploit) or try something new that might be better (explore)? Previous research has identified two distinct styles of human exploration:
- Directed Exploration: Strategically choosing options with high uncertainty to gain information (often modeled as an "uncertainty bonus").
- Random Exploration: Adding stochastic "noise" to choices to ensure a variety of options are sampled.
While the speed-accuracy trade-off is well-documented, it was previously unclear how time pressure affects these specific strategies. Does a ticking clock simply make us messy and random, or do we adapt by ditching the most "expensive" mental calculations? This study investigates whether the cognitive cost of processing uncertainty leads to a "resource-rational" shift in strategy under pressure.
Methods
The researchers recruited 99 participants for an online "Time Bandit" experiment consisting of several four-armed bandit tasks. Participants chose between four options (keys Q, W, O, P), each yielding rewards from different underlying distributions.
The study employed a within-subjects design with two primary manipulations:
- Time Pressure: "Unlimited" time vs. "Limited" time (decisions required in <400ms).
- Payoff Structures: Four conditions (IGT-like, Low Variance, High Variance, and Equal Means) designed to decouple the influence of expected reward from the influence of uncertainty.
To analyze the data, the authors used a Bayesian Mean Tracker (BMT) to estimate how participants learned reward means and uncertainties. Choice behavior was modeled using a hierarchical Softmax model to quantify sensitivity to value ($\alpha$), uncertainty bonuses ($\beta$), and perseveration ($\gamma$). Additionally, a Linear Ballistic Accumulator (LBA) model decomposed reaction times (RT) into drift rates (speed of information processing) and decision thresholds (how much evidence is needed to commit).
Key Results and Insights
The study found that uncertainty is mentally "expensive." In unlimited time conditions, participants took longer to choose options with high uncertainty, suggesting that integrating uncertainty into a decision requires active, time-consuming computation.
Under time pressure, participants didn't just fail; they adapted through a resource-rational shift:
- Strategy Shift: Participants showed a significant decrease in the uncertainty bonus ($\beta$) and a decrease in value-directedness ($\alpha$). When time is scarce, the "uncertainty bonus" is the first thing to go.
- Increased Perseveration: The odds of repeating a choice were 1.4 times higher under time pressure, suggesting that "stickiness" is a low-cost fallback strategy when cognitive resources are overtaxed.
- Acceleration vs. Caution: The LBA model revealed that time pressure led to lower decision thresholds (deciding with less evidence) but higher drift rates. This indicates that people "accelerate" their information processing to meet the deadline.
- RT Dynamics: The "uncertainty slowdown" observed in unlimited time was significantly dampened under pressure, confirming that participants were ignoring uncertainty to save time.
Discussion and Future Work
These results demonstrate that humans strategically simplify their search for rewards when the clock is ticking. By reducing uncertainty-directed exploration and increasing perseveration, decision-makers minimize cognitive load at the expense of long-term information gain.
However, several questions remain. It is unclear if time pressure also impairs the learning process (forming mental maps) or if it primarily affects the choice process. Furthermore, the task only involved positive reinforcement; exploration behavior often changes significantly in the domain of losses. Future work could examine if these strategy shifts hold in more complex, structured environments where directed exploration is even more valuable.
Reference: Wu, C.M., Schulz, E., Pleskac, T.J. et al. Time pressure changes how people explore and respond to uncertainty. Sci Rep 12, 4122 (2022). https://doi.org/10.1038/s41598-022-07901-1
Summary updated