Adaptive Uncertainty Quantification for Heteroscedastic Noise

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2249 papers350 conceptsfrozen 2026-10-04

In standard regression, we typically ask a model to predict a single value $y$ for a given input $x$. When we do ask for uncertainty, most classical models provide a "one-size-fits-all" error bar. This assumes homoscedasticity: the belief that the noise level is constant across the entire input space.

However, real-world data is rarely so uniform. In finance, engineering, or fluid dynamics, some regions of the input space are inherently more volatile than others. This phenomenon is known as heteroscedasticity. If a model ignores these local variations, it becomes dangerously overconfident in high-noise regions and unnecessarily conservative in low-noise ones.

Recent research has converged on a shared goal: moving from global error estimates to localized, adaptive uncertainty quantification that "learns" the noise surface alongside the prediction.

The Failure of Global Error Bars

The core problem with homoscedastic assumptions is that they treat all errors as equal. As David A. Nix and Andreas S. Weigend pointed out in their foundational work, Learning Local Error Bars for Nonlinear Regression, standard models are often overly sensitive to outliers in high-noise zones. By explicitly modeling the target as a conditional distribution—where the variance is itself a function of the input—the system becomes more robust.

When a model fails to account for these local variations, it cannot distinguish between a "reliable" prediction and one prone to significant error. This is particularly critical in fields like bioprocessing, where Tien Pham and colleagues show that assuming uniform noise can lead optimization algorithms to converge on solutions that appear robust in simulation but fail under real-world biological fluctuations Single-objective robust optimisation in bioprocessing.

Learning the Noise Surface

To capture heteroscedasticity, models must treat variance as a learnable parameter. Mark Hamilton argues in It Is Likely That Your Loss Should be a Likelihood that we should move away from rigid loss functions like Mean Squared Error. By optimizing the full likelihood, the model treats variance as a variable, allowing it to "de-weight" noisy data points automatically.

This "learning the noise" approach takes several forms across different architectures:

Localizing Conformal Prediction

While Bayesian methods provide elegant probabilistic frameworks, Conformal Prediction (CP) offers rigorous statistical guarantees on error rates without assuming a specific data distribution. However, standard CP is often "global," applying the same interval width to every point.

To make CP adaptive, researchers are now "reweighting" it based on local difficulty. Salim I. Amoukou and Nicolas J-B. Brunel use Quantile Regression Forests to locally scale prediction intervals, ensuring they are tight where the model is confident and wide where it is not Adaptive Conformal Prediction by Reweighting Nonconformity Score. Similarly, Etash Kumar Guha and Shlok Natarajan discretize the output space into bins to handle multimodal or highly heteroscedastic data, producing tighter sets than traditional regression baselines Conformal Prediction via Regression-as-Classification.

Geometry and Proximity

Another intuition for local uncertainty comes from the "internal geometry" of the model itself. For instance, in Random Forests, we can look at how often data points end up in the same leaf nodes. Jake S. Rhodes and colleagues leverage these "proximities" to create localized prediction intervals that outperform global methods by focusing on the errors of "nearby" points in the feature space Localized Uncertainty Quantification in Random Forests via Proximities.

This spatial awareness is also vital in physical modeling. Eric Crislip notes that uniform noise assumptions fail in dynamical systems with sharp gradients; future models must use spatially varying scale parameters to avoid underestimating uncertainty in complex regions Closure Term Estimation in Spatiotemporal Models of Dynamical Systems.

Impact on Bayesian Optimization

Perhaps the most practical application of this research is in Bayesian Optimization (BO). Standard BO solvers often struggle when hyperparameter landscapes have varying noise levels—for example, a model's performance might be very stable at low learning rates but highly volatile at high ones.

The HEBO framework, developed by Alexander I. Cowen-Rivers and others, won the NeurIPS 2020 Black-Box Optimization challenge by explicitly accounting for this heteroscedasticity HEBO: An Empirical Study of Assumptions in Bayesian Optimisation. By warping inputs and outputs to handle non-stationary noise, HEBO avoids the pitfalls of "clean" assumptions that lead other optimizers to make suboptimal sampling decisions in volatile regions.

Papers in this version

Showing 60 of 249 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 192 papers
  2. Version 1 · 2026-10-04 · 249 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_heteroscedastic_noise_modeling_input_dep_aff54309,
  title = {heteroscedastic noise modeling (input-dependent noise modeling)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/heteroscedastic-noise-modeling-input-dependent-noise-modeling/rdr_aff543095ed448c5becf1cd65567dba9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_adaptive_uncertainty_quantification_for__aff54309_v1,
  title = {Adaptive Uncertainty Quantification for Heteroscedastic Noise},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adaptive-uncertainty-quantification-for-heteroscedastic-noise/rdr_aff543095ed448c5becf1cd65567dba9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}