Asymmetric Learning Rates in Biological and Artificial Agents

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 264 papers104 conceptsfrozen 2026-10-04

In classical reinforcement learning (RL), an agent learns by comparing what it expected to happen with what actually happened. This difference is the Reward Prediction Error (RPE). Standard models, like the Delta rule, assume we learn from surprises at a constant speed: if you win \$10 unexpectedly, you update your internal "value" of that action by the same magnitude as if you had unexpectedly lost \$10.

However, a growing body of research across neuroscience, psychiatry, and artificial intelligence suggests that both biological brains and high-performing AI agents are fundamentally asymmetric learners. Instead of a single learning rate, these systems employ distinct rates for positive and negative surprises. This asymmetry is not a cognitive "glitch" but a sophisticated mechanism for managing risk, navigating uncertainty, and representing the full distribution of possible outcomes.

The Mechanism of Valence-Specific Updating

The core idea is a simple modification to the standard update rule. Instead of one learning rate ($\alpha$), we use two: $\alpha_+$ for positive prediction errors (better than expected) and $\alpha_-$ for negative ones (worse than expected).

As Yael Niv, Peter Dayan, and colleagues demonstrated in Neural Prediction Errors Reveal a Risk-Sensitive Reinforcement-Learning Process in the Human Brain, this simple split allows a learner to be inherently risk-sensitive. If $\alpha_- > \alpha_+$, the agent is "pessimistic" or risk-averse, weighting unexpected losses more heavily than gains. Conversely, if $\alpha_+ > \alpha_-$, the agent becomes "optimistic" or risk-seeking.

This isn't just about how much we like a reward (utility); it’s about the speed at which we update our beliefs. A risk-averse person doesn't just fear loss; they learn from a single failure much faster than they learn from a single success.

Distributional RL and the Diversity of Neurons

Why would the brain maintain multiple learning rates simultaneously? Recent work on Distributional Reinforcement Learning provides a compelling answer. Rather than just tracking the average expected reward, the brain appears to represent the entire probability distribution of potential outcomes.

In Distributional reinforcement learning in prefrontal cortex, researchers found that neurons in the anterior cingulate cortex (ACC) specialize. Some neurons are "optimistic" (high $\alpha_+$, low $\alpha_-$), while others are "pessimistic." By pooling these biased updates, the brain can "see" the full range of risks and rewards. Will Dabney argues in Depression as a disorder of distributional coding that clinical depression may actually be the result of a "dropout" of these optimistic units, leaving the brain with a skewed, pessimistic view of the world.

Asymmetry as a Rational Strategy

While "bias" often sounds negative, asymmetric learning is frequently the most rational way to solve complex problems.

  1. Exploration-Exploitation: Johannes A. Schubert and colleagues showed in In-Context Learning Agents Are Asymmetric Belief Updaters that Large Language Models (LLMs) naturally develop an "optimism bias" during in-context learning. This bias helps the agent stay engaged with potentially high-value actions rather than giving up too early after a single negative result.
  2. Environmental Richness: In "poor" environments where rewards are rare, a positivity bias (learning more from the few wins you get) is often optimal for distinguishing between mediocre and truly bad options, as explored by Henri Vandendriessche in Contextual influence of reinforcement learning performance of depression.
  3. Market Volatility: In financial AI, Xinyi Li demonstrated that agents using asymmetric updates (Optimistic Bull or Pessimistic Bear) outperform standard models by adapting their sensitivity to bullish or bearish signals separately.

Clinical Phenotyping: When Asymmetry Goes Awry

By treating these learning rates as "knobs" that can be tuned, researchers can create mathematical "fingerprints" for mental health conditions. This is known as computational phenotyping.

  • Anxiety: Linked to a "negativity bias" in updating, where individuals over-weight threat-related information (Wise et al., 2019).
  • Depression: Characterized by a failure to utilize positive feedback, particularly in low-reward contexts, or a global shift toward learning more from failures than successes (Gagne et al., 2022).
  • Sleep Deprivation: Wenhao Xu found that sleep loss specifically impairs the ability to learn from negative feedback ($\alpha_-$ drops), while the response to rewards remains stable, leading to dangerously risky choices (Sleep deprivation alters utilization of negative feedback in risky decision-making).

From Biology to AI

The transition from symmetric to asymmetric learning represents a shift from "perfectly logical" agents to "ecologically rational" ones. Whether it's a mouse learning to avoid a shock, a human navigating a social hierarchy, or a Deep Q-Network landing a spacecraft (CM-DQN), the ability to weight "good news" and "bad news" differently is a fundamental requirement for surviving and thriving in an uncertain world.

By mimicking these biological asymmetries, AI researchers like Grégoire Delétang are building agents that don't just maximize numbers, but exhibit human-like risk-sensitivity and robustness (Model-Free Risk-Sensitive Reinforcement Learning).

Papers in this version

Showing 60 of 64 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 107 papers
  2. Version 1 · 2026-10-04 · 64 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_asymmetric_learning_rates_asymmetric_rei_4a343d15,
  title = {asymmetric learning rates (asymmetric reinforcement learning)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asymmetric-learning-rates-asymmetric-reinforcement-learning/rdr_4a343d15063347f893fc4a81b2d63998},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_asymmetric_learning_rates_in_biological__4a343d15_v1,
  title = {Asymmetric Learning Rates in Biological and Artificial Agents},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asymmetric-learning-rates-in-biological-and-artificial-agents/rdr_4a343d15063347f893fc4a81b2d63998/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}