Abstract
In many real-world scenarios, exploration is not free; a single bad choice can lead to catastrophic failure. This study explores how humans navigate these "safe optimization" problems—tasks where an unknown function must be maximized while remaining above a critical danger threshold. Through behavioral experiments and computational modeling using Gaussian Processes, we demonstrate that humans are "safe-first" optimizers. Rather than actively expanding their knowledge of safe territories, participants prioritize staying within a perceived safe set, favoring local exploitation over global exploration.
Introduction
The "exploration-exploitation" trade-off is a central theme in machine learning and cognitive science. While traditional models often assume that every choice provides a reward, real-world environments frequently impose hard constraints. Foragers, engineers, and investors must optimize outcomes while avoiding thresholds that lead to total failure, such as poisoning, structural collapse, or bankruptcy. This paper investigates the cognitive strategies humans employ when faced with such "deadly" exploration. Specifically, we ask whether humans actively expand their safe territory or simply optimize within a known "safe bubble."
Methods
The researchers conducted two behavioral experiments via Amazon Mechanical Turk. In Experiment 1 (Univariate), participants interacted with 1D functions where they aimed to find the peak while staying above a red danger line. If a participant's choice fell below this threshold, the trial block ended immediately. Experiment 2 (Bivariate) involved participants navigating 2D grids, comparing behavior in "normal" (no threshold) versus "safe" (with threshold) conditions.
To analyze behavior, the authors utilized Gaussian Process (GP) regression to model human function learning. They compared human choices against a Safe-Optimization algorithm that partitions potential actions into three categories: a Safe Set ($S_t$) of points likely above the threshold, Maximizers ($M_t$) likely to be the global maximum, and Expanders ($G_t$) likely to reveal new safe areas.
Results and Key Insights
Participants demonstrated high performance, significantly outperforming chance despite the inherent risks. However, their behavior deviated sharply from standard machine learning "Safe-Opt" algorithms.
The "Safe-First" Heuristic
Analysis revealed that humans do not treat safety and reward as a single integrated value. Instead, they employ a two-step heuristic: first, filter for points with a very high probability of being safe ($p > 0.99$), and then select the best-looking point within that subset. This behavior was accurately captured by a simple decision tree model.
Ignoring Expanders and Local Search
Unlike optimal algorithms that prioritize "expanders" to grow the safe zone, humans almost never actively scout the boundaries. Choices were highly localized, staying close to the starting point. This suggests a strategy geared toward homeostasis—maintaining a safe state—rather than absolute optimization. Interestingly, a "shadow of risk" was observed in the 2D experiment: even when the safety threshold was removed, participants continued to behave as if a danger zone existed, suggesting a strong generalization of risk-aversion.
Discussion
The study highlights that human optimization is fundamentally constrained by a "safety-first" priority. This risk-averse behavior mirrors biological imperatives where survival takes precedence over maximal gain. While the current experiments used short horizons (10 trials), future research could explore whether longer interactions or different environmental "smoothness" might eventually encourage more daring exploration. Understanding the "breaking point" where humans transition from safety-seeking to risk-taking remains a critical open question for future study.
Summary updated