Non-Linear Probability Weighting for Risk-Aware Decision Models

293 concepts138 papers1 proposal

Standard decision-making frameworks in machine learning and economics typically operate on a simple premise: maximize the expected value. To find the optimal action, an agent multiplies the objective probability of an outcome by its utility and sums them up.

However, this linear treatment of probability fails to capture how humans actually perceive risk, and it frequently falls short when autonomous systems must operate safely in volatile, real-world environments. If a system optimizes purely for the average case, it may accept a tiny, 0.1% risk of a catastrophic failure if the 99.9% upside is high enough. As Leqi et al. observe in On Human-Aligned Risk Minimization, traditional loss functions treat errors of equal magnitude identically regardless of their frequency, completely missing the human tendency to prioritize avoiding rare, extreme losses.

To bridge this gap, a growing body of research integrates non-linear probability weighting—a core mechanism from Cumulative Prospect Theory (CPT)—into machine learning, reinforcement learning, and game theory. By distorting objective probabilities into subjective decision weights, these models align artificial intelligence with the psychological realities of human risk perception.


The Intuition: The Inverse-S Curve

Why do humans buy lottery tickets while simultaneously purchasing flight insurance? Under expected utility theory, these behaviors contradict each other. The answer lies in how we distort likelihoods: humans systematically overweight rare events (low probabilities) and underweight highly certain events (high probabilities).

Subjective Weight w(p)
  1.0 |          .---'
      |        ./
      |       /
      |      /  <-- Inverse-S Curve
      |    ./
      |  ./
  0.0 +-----------------
     0.0             1.0
     Objective Probability p

Mathematically, this distortion is captured by mapping an objective probability $p$ to a subjective weight $w(p)$ using an inverse-S curve.

  1. At the low end ($p \to 0$): The curve is steep. A 1% chance of a catastrophic data breach or a massive financial loss is psychologically inflated, causing agents to overreact.
  2. At the high end ($p \to 1$): The curve flattens. A 99% chance of success feels distinctly uncertain, leading to risk-averse behavior.

By replacing linear expectations with these distorted weights, decision models can simulate or interact with "boundedly rational" human agents. As Gopalan et al. demonstrate in Weighted Bandits or: How Bandits Learn Distorted Values That Are Not Expected, when learning agents ignore these perceptual distortions, they optimize for objectives that humans find fundamentally undesirable or suboptimal.


Risk-Aware Reinforcement Learning and Bandits

Integrating non-linear probability weighting into sequential decision-making fundamentally changes how agents evaluate exploration and safety. Instead of calculating standard expected returns, agents optimize a prospect-theoretic objective where tail-end risks are amplified.

This transformation introduces significant mathematical challenges because non-linear weighting functions warp the optimization landscape, making it non-convex and breaking standard Bellman optimality. To resolve this, Lepel et al. derived a first-order policy gradient theorem tailored for CPT in Beyond Expected Returns: A Policy Gradient Algorithm for Cumulative Prospect Theoretic Reinforcement Learning, providing a scalable way for agents to learn under distorted probability landscapes.

When agents are trained with these subjective distortions, they exhibit a natural capacity for hazard avoidance. For example, Rajabi et al. developed a privacy-preserving reinforcement learning framework in Privacy-Preserving Reinforcement Learning Beyond Expectation, demonstrating that agents incorporating non-linear probability weighting hit high-penalty obstacles significantly less often than risk-neutral agents. The inflation of low-probability negative events acts as an inherent mathematical safety buffer.

This framework extends across various bandit and exploration settings:


Predicting and Collaborating with Humans

When robots and AI systems collaborate with humans, assuming the human is a perfectly rational utility-maximizer leads to coordination failures. Humans frequently make choices that appear suboptimal under standard probability theory but are entirely consistent when viewed through the lens of cognitive biases.

In human-robot interaction, Biyik et al. showed in When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humans that integrating CPT into a robot's motion planner drastically improves its ability to anticipate human actions. Because humans overweight low-probability risks, they often choose longer, safer trajectories. A robot that models this non-linear perception can proactively yield or adapt, ensuring smoother and safer collaboration.

Similarly, in explainable AI and dialogue systems, Tang et al. demonstrated in Approximating Human Models During Argumentation-based Dialogues that human trust and certainty do not scale linearly. By transforming raw confidence scores using a prospect-theoretic probability weighting function, an AI agent can accurately track a user's internal mental state and adjust its explanations to match the user's actual psychological certainty.


Reshaping Game Theory and Strategic Behavior

In multi-agent environments, replacing linear probabilities with subjective weights fundamentally alters the landscape of strategic equilibria.

In traditional game theory, predicting outcomes with complex, asymmetric agents can lead to an unmanageable explosion of possible Nash equilibria. However, Krishnan K. S. et al. proved in How Irrationality Shapes Nash Equilibria: A Prospect-Theoretic Perspective that introducing non-linear value and weighting functions can actually simplify games by "pruning" away unstable, messy equilibria, collapsing infinite outcome sets into predictable points.

Conversely, ignoring these distortions can expose systems to severe vulnerabilities. In security contexts, Sanjab et al. analyzed drone delivery networks in Prospect Theory for Enhanced Cyber-Physical Security of Drone Delivery Systems: A Network Interdiction Game, revealing that human operators' tendency to underweight high-probability risks can lead them to select dangerous flight paths that a purely objective model would flag as unviable.

This insight is also critical for machine learning models deployed in adversarial settings. Lv et al. introduced a prospect-guided framework for strategic classification in Beyond Rational Illusion: Behaviorally Realistic Strategic Classification. They demonstrated that when humans attempt to "game" a classifier (such as modifying features to pass a credit screening), they evaluate their probability of success non-linearly. Classifiers that explicitly model this inverse-S distortion prove significantly more robust in real-world deployments than those assuming perfect mathematical rationality.


Why the Distortion Works: An Algorithmic Feature

A lingering question is whether non-linear probability weighting is simply an evolutionary "bug" or if it serves an underlying functional purpose. Recent computational neuroscience and algorithmic analyses suggest that these distortions are a highly efficient adaptation to bounded resources.

Tong et al. demonstrated in The Bayesian Origin of the Probability Weighting Function in Human Representation of Probabilities that the classic inverse-S weighting function emerges naturally when an optimal Bayesian observer decodes noisy internal signals, provided that finite neural encoding resources are prioritized near the boundaries of absolute certainty ($0$ and $1$).

Furthermore, from a purely algorithmic standpoint, Salehi Nobandegani et al. showed in The St. Petersburg Paradox: A Fresh Algorithmic Perspective that applying sublinear transformations to payoffs prevents extreme, low-probability outliers from dominating computational decision processes. What appears to be cognitive irrationality under standard linear models is, in reality, a manifestation of algorithmic efficiency under finite computational constraints.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 158 papers
  2. Version 1 · 2026-10-04 · 298 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_non_linear_probability_weighting_2748ba37,
  title = {non-linear probability weighting},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-linear-probability-weighting/rdr_2748ba372cad4fa6afd7691c8fa85c03},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_non_linear_probability_weighting_2748ba37_v2,
  title = {non-linear probability weighting},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-linear-probability-weighting/rdr_2748ba372cad4fa6afd7691c8fa85c03/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake