Robust Learning via Uncertainty Weighted Regression

This direction is merged into another direction: adversarial corruption robustness. Its last version stays readable and citable here.

Version 1 of 118 papers29 conceptsfrozen 2026-10-04

In traditional reinforcement learning and bandit problems, uncertainty is an invitation. Under the principle of "Optimism in the Face of Uncertainty," an agent treats a lack of data as a reason to explore, assigning high value to unknown states to see if they hold hidden rewards. However, in the presence of an adversary, this curiosity becomes a critical vulnerability.

Recent research across contextual bandits and reinforcement learning (RL) has identified a shared solution to this problem: Uncertainty Weighting. By treating uncertainty with optimism during action selection but with skepticism during model training, researchers have developed a robust framework that prevents "poisoned" data from derailing the learning process.

The Vulnerability of the Unknown

To understand why uncertainty is dangerous, consider how a model learns. In a standard regression task, every data point is usually treated as equally valid. If an adversary wants to manipulate a model, they will target the regions where the model is most "flexible"—the areas where it has the least data.

In these high-uncertainty regions, a single corrupted reward or transition can significantly shift the model's parameters. As Dongruo Zhou, Tong Zhang, and Quanquan Gu demonstrate in Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial Corruptions, an attacker can exert massive leverage by providing extreme rewards in states where the agent’s exploration bonus is high. Without a defense, the cost of this corruption scales multiplicatively with the total regret, quickly making the model useless.

The Duality of Uncertainty

The core insight of this research direction is a fundamental duality: uncertainty should be used for exploration, but it must be de-emphasized for estimation.

In Corruption-Robust Algorithms with Uncertainty Weighting for Nonlinear Contextual Bandits and Markov Decision Processes, Chenlu Ye and Wei Xiong propose a weighted least-squares (or Maximum Likelihood Estimation) approach. Instead of treating all samples equally, the learner assigns a weight to each observation that is inversely proportional to its predictive uncertainty (often represented by a confidence radius or information ratio).

This mechanism acts as a filter. When the model encounters a state it knows well, it trusts the feedback. When it encounters a state where it is highly uncertain—the exact place an adversary would strike—it down-weights that feedback. This prevents a small number of heavily corrupted samples from dominating the regression objective.

Neutralizing Adversarial Leverage

Why does this specific weighting work so well? It effectively caps the "influence" of any single data point.

In the context of linear bandits, Gagandeep Singh and Zhiwei Wang show in Robust Thompson Sampling Algorithms Against Reward Poisoning Attacks that attackers often use high-magnitude feature vectors to exert leverage. By down-weighting these high-influence points, the model bounds the potential bias an attacker can introduce.

This approach has been successfully extended to several complex domains:

From Multiplicative to Additive Regret

The mathematical "win" of this direction is the shift from multiplicative to additive regret. In older robust algorithms, the presence of corruption would multiply the entire regret term, meaning the model's performance would degrade significantly even with small amounts of noise.

By using uncertainty weighting, researchers like Zhiyong Wang and Shuai Li have shown that the cost of corruption can be isolated. In their work on Online Corrupted User Detection and Regret Minimization, the regret scales linearly with the total amount of corruption. This means the model remains efficient in clean environments while degrading gracefully—rather than catastrophically—under attack.

By integrating skepticism directly into the loss function, these researchers have moved beyond simple outlier detection. They have created a way for models to learn from the world while remaining fundamentally wary of the unknown.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_robust_learning_via_uncertainty_weighted_3b7e5a52,
  title = {Robust Learning via Uncertainty Weighted Regression},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-learning-via-uncertainty-weighted-regression/rdr_3b7e5a52d6fb43a3b87760827e67504c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_robust_learning_via_uncertainty_weighted_3b7e5a52_v1,
  title = {Robust Learning via Uncertainty Weighted Regression},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-learning-via-uncertainty-weighted-regression/rdr_3b7e5a52d6fb43a3b87760827e67504c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}