Probabilistic Numerical Methods for Integration and Differential Equations

74 concepts52 papers2 proposals

Every numerical algorithm confronts a fundamental information deficit. When calculating an integral or solving a differential equation, a computer can only evaluate a function at a finite number of points. Because infinitely many continuous functions can pass through those same points, the exact solution remains underdetermined. Classical numerical analysis sweeps this underdetermined nature under the rug, returning a single deterministic point estimate.

Probabilistic Numerics reframes this challenge. Instead of viewing numerical error as a deterministic value to be bounded by worst-case analysis, it treats the unknown function as a hidden variable to be inferred. By placing a statistical prior over the function space and treating evaluations as data, numerical tasks become problems of Bayesian inference. This paradigm shift allows algorithms to output a full posterior distribution, providing both a best guess and a calibrated measure of epistemic uncertainty.

Numerical Integration as Gaussian Process Inference

To see how this works in practice, consider the task of numerical integration. In Bayesian Monte Carlo, Carl Edward Rasmussen et al. proposed replacing traditional "blind" sampling with a principled statistical framework known as Bayesian Quadrature.

Instead of treating the integrand $f$ as a completely unknown black box, we encode our prior beliefs about its properties—such as smoothness or scale—by modeling it as a Gaussian Process (GP). When we evaluate the function at a finite set of design points, these evaluations serve as data. Conditioning the GP on these observations yields a posterior distribution over the function space.

Because integration is a linear operator, applying it to a Gaussian Process yields another Gaussian distribution. Crucially, this means we can analytically compute a full posterior distribution over the value of the integral itself. As discussed by Francois-Xavier Briol et al. in Rejoinder: Probabilistic Integration: A Role in Statistical Computation? and Mark Girolami et al. in Editorial: special edition on probabilistic numerics, the posterior mean provides the optimal estimate given our prior, while the posterior variance explicitly quantifies the discretization error resulting from finite sampling.

The Deep Connection to Classical Methods

This probabilistic view does not abandon classical numerical analysis; rather, it unifies and generalizes it. Traditional quadrature rules estimate integrals via a weighted sum of function evaluations, $\sum w_i f(x_i)$. In the probabilistic framework, these weights emerge naturally: the posterior mean of a GP conditioned on the samples is mathematically equivalent to an optimally weighted estimator.

For example, Manuel Graef et al. show in Numerically Optimizing Weights for Monte Carlo Integration on Smooth Compact Manifolds that weight optimization on complex geometries is equivalent to Bayesian cubature. This connection is expanded by Martin Ehler et al. in Optimal Monte Carlo integration on closed manifolds, proving that interpreting integration as GP regression yields near-optimal convergence rates on manifolds.

To bridge the gap between Bayesian flexibility and the rigid robustness of classical polynomial-exact rules, the authors of A Bayes-Sard Cubature Method introduce an improper flat prior on the coefficients of a regression-based mean function. This mathematical choice forces the resulting integration rule to be exactly correct for any function within the span of a chosen polynomial basis, protecting the solver against poorly chosen kernel parameters while still providing a valid variance envelope. Furthermore, single-author work by Francois-Xavier Briol in Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees establishes that integrating greedy point selection with these optimal weighting schemes guarantees exponential and super-exponential posterior contraction rates, proving that probabilistic integrators can match or exceed the convergence properties of classical methods.

Solving Differential Equations via Bayesian Filtering

The leap from integration to solving ordinary differential equations (ODEs) follows the same underlying logic but introduces a dynamic twist. An ODE defines a relationship between an unknown curve and its derivatives. Philipp Hennig et al. in Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics demonstrated that this can be framed as a joint GP regression task: a prior is placed over the solution curve, and evaluations of the vector field serve as derivative observations.

To make this computationally tractable, David Duvenaud et al. developed Gauss-Markov-Runge-Kutta solvers in Probabilistic ODE Solvers with Runge-Kutta Means. By utilizing $s$-times integrated Wiener processes as priors, they constructed probabilistic solvers whose posterior means match the exact point estimates of classical Runge-Kutta methods. Crucially, these solvers use probabilistic continuation to maintain a continuous belief distribution across steps, allowing the model to capture how uncertainty accumulates globally over time. Similarly, Onur Teymur et al. established in Probabilistic Linear Multistep Methods that classical multistep methods can be recovered as the posterior mean of a specific joint GP model over the solution and its derivatives.

A highly intuitive way to conceptualize this is through the lens of state-space modeling and filtering. As formulated by Filip Tronarp et al. in Probabilistic solutions to ordinary differential equations as nonlinear Bayesian filtering: a new perspective, we can treat the ODE residual as a "zero-measurement" in a state-space model. This formulation decouples the prior belief about the solution's smoothness from the physical constraints of the vector field, transforming the ODE solver into a standard Bayesian filtering and smoothing agent.

Why Probabilistic Numerics Matters

Why should we care about treating numerical methods probabilistically? In isolation, a small discretization error from a standard solver might seem harmless. However, modern scientific computing rarely uses a solver in isolation; instead, solvers are embedded inside complex, multi-layered pipelines like state estimation, optimal control, or parameter optimization. In these settings, ignoring numerical error can lead to catastrophic failures or highly overconfident, incorrect conclusions.

Consider the domain of non-linear state estimation. In Improved Calibration of Numerical Integration Error in Sigma-Point Filters, Jakub Prüher et al. demonstrate that standard sigma-point filters frequently diverge because they treat approximate numerical integration as exact. By explicitly modeling the variance of the integration error through a hierarchical GP prior, the filter maintains well-calibrated state estimates and avoids overconfidence.

In biological simulations, Nicholas Krämer et al. show in Probabilistic solvers enable a straight-forward exploration of numerical uncertainty in neuroscience models that deterministic ODE solvers can completely hide integration artifacts, such as shifted or entirely missing action potentials. By generating a distribution of possible trajectories, probabilistic solvers expose these artifacts, ensuring that scientific conclusions reflect actual biology rather than numerical noise.

Finally, this calibrated uncertainty can be actively leveraged during optimization. In Probabilistic ODE solvers for integration error-aware numerical optimal control, Amon Lahr et al. substitute deterministic solvers with probabilistic ones in optimal control loops. Because the solver accounts for the covariance of the state estimate, the controller can optimize for trajectories that minimize both operational costs and numerical uncertainty, leading to robust performance even under tight computational budgets.

Key Foundational Work

Bayesian Monte Carlo (2002) — contributed 2 concept elements to this direction.
Non parametric Bayesian drift estimation for one-dimensional diffusion processes (2009) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_probabilistic_numerical_integration_5c53e368,
  title = {probabilistic numerical integration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/probabilistic-numerical-integration/rdr_5c53e36827c44bd8a34c4f2c4e78d910},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_probabilistic_numerical_integration_5c53e368_v1,
  title = {probabilistic numerical integration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/probabilistic-numerical-integration/rdr_5c53e36827c44bd8a34c4f2c4e78d910/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake