Affine Invariant Langevin Dynamics for rare-event sampling

Deepyaman Chakraborty, R. M. Harris, Rupert Klein, Guillermo Olicón-Méndez, Sebastian Reich, Claudia Schillings

TL;DR

This paper introduces ALDI (Affine Invariant Langevin Dynamics), a robust framework for sampling rare events in complex dynamical systems. By reframing rare-event estimation as a Bayesian inverse problem and leveraging affine invariance, the method efficiently "finds" low-probability regions (like atmospheric blockings) that traditional Monte Carlo methods would miss.

What Problem Does This Paper Address?

Rare events—such as extreme weather patterns or transitions in molecular states—are by definition infrequent, yet they carry immense consequences. For researchers, they are a nightmare to simulate: because their probability is exponentially small, a "crude" Monte Carlo simulation might need to run for millions of years before seeing a single event.

Existing specialized tools have drawbacks. Importance Sampling requires a good "guess" (proposal distribution) of where the event happens, which is hard to find in high dimensions. The Ensemble Kalman Filter (EnKF) is popular but can struggle with non-Gaussian distributions or "collapse," where all particles end up in the same spot, failing to represent the full range of possibilities. This paper seeks a method that is derivative-free, robust to complex geometries, and mathematically guaranteed to converge.

Methods

The authors propose a two-step strategy:

  1. Bayesian Reformulation & Smoothing: They define the rare event using a "limit-state function" $G(x)$, where the event occurs if $G(x) \le 0$. To make this mathematically tractable for dynamics, they introduce a smooth approximation ($\tilde{G}_\delta$) and treat the search for the "failure set" as a Bayesian inverse problem with a "small-noise" limit.
  2. ALDI (The Sampler): They use Affine Invariant Langevin Dynamics to sample the resulting posterior. ALDI is an interacting particle system. Its "secret sauce" is affine invariance: the algorithm's performance is unaffected by linear rescaling or "stretching" of the coordinate system. This allows the particles to automatically adapt to the local "shape" (anisotropy) of the rare-event region without manual tuning.
  3. Importance Sampling (IS) Correction: Since the ALDI sampler works on a smoothed version of the problem, the authors use the resulting particle cloud to fit a Gaussian Mixture Model. This mixture serves as a highly informed proposal distribution for a final Importance Sampling step, which provides the actual probability estimate.

Datasets

The method was evaluated on a hierarchy of synthetic benchmarks designed to mimic real-world challenges:

  • Convex Limit-State Function: A 2D algebraic problem used to test the basic geometry-handling capabilities.
  • Hyperbolic Saddle: A 2D linear ODE system with an unstable equilibrium, representing systems where trajectories rapidly diverge.
  • Point-Vortex Model: A 6D system (representing three interacting vortices) used as a proxy for atmospheric blockings—persistent high-pressure systems that "block" normal air flow and cause heatwaves or floods.

Key Insights and Ideas

  • The "Aha" Moment on Convergence: The authors prove that as the "observation noise" in their Bayesian formulation goes to zero, the estimator converges to the true rare-event probability, regardless of how much smoothing was applied to the limit-state function.
  • Geometry Adaptation: Unlike standard Langevin methods that might struggle if a distribution is long and thin, ALDI’s affine invariance allows the particle ensemble to "feel" the covariance of the target and adjust its step sizes accordingly.
  • Stability over Direct Estimation: The paper highlights that trying to estimate the probability directly from the ALDI particles (Variant 1) is numerically unstable. Instead, using ALDI to "find the neighborhood" and then performing Importance Sampling (Variant 2) is far more robust.
  • Gradient-Free Potential: The framework includes a gradient-free version, which is vital for complex simulations where calculating exact derivatives of the model is impossible or too expensive.

Key Results

  • Efficiency: In the "rarer-event" regimes where crude Monte Carlo failed to find any samples, ALDI-based Importance Sampling provided stable estimates with controlled variance.
  • Accuracy: For the point-vortex model, ALDI successfully concentrated particles near the "near-critical" sets (the configurations leading to blockings), yielding accurate proposal distributions even in highly unstable regimes.
  • Robustness: The error consistently decreased as the ensemble size ($J$) increased, confirming the method scales predictably.

Limitations and Future Work

  • Theoretical Gaps: While the gradient-based ALDI is well-understood, a full theoretical analysis of the gradient-free version remains an open problem.
  • High-Dimensional Scaling: While tested on 6D systems, the computational cost for very high-dimensional systems (like full-scale climate models) needs further investigation.
  • Future Directions: The authors suggest extending the framework to fully time-dependent PDEs and developing "multi-level" variants to reduce the cost of the underlying forward model evaluations.

Summary updated

Abstract

We introduce an affine invariant Langevin dynamics (ALDI) framework for the efficient estimation of rare events in nonlinear dynamical systems. Rare events are formulated as Bayesian inverse problems through a nonsmooth limit-state function whose zero level set characterises the event of interest. To overcome the nondifferentiability of this function, we propose a smooth approximation that preserves the failure set and yields a posterior distribution satisfying the small-noise limit. The resulting potential is sampled by ALDI, a (derivative-free) interacting particle system whose affine invariance allows it to adapt to the local anisotropy of the posterior. We demonstrate the performance of the method across a hierarchy of benchmarks, namely two low-dimensional examples (an algebraic problem with convex geometry and a dynamical problem of saddle-type instability) and a point-vortex model for atmospheric blockings. In all cases, ALDI concentrates near the relevant near-critical sets and provides accurate proposal distributions for self-normalised importance sampling. The framework is computationally robust, potentially gradient-free, and well-suited for complex forward models with strong geometric anisotropy. These results highlight ALDI as a promising tool for rare-event estimation in unstable regimes of dynamical systems.
BibTeX
@misc{chakraborty2025affine,
  author = {Deepyaman Chakraborty and R. M. Harris and Rupert Klein and Guillermo Olicón-Méndez and Sebastian Reich and Claudia Schillings},
  title = {{Affine Invariant Langevin Dynamics for rare-event sampling}},
  year = {2025},
  eprint = {2601.00107},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.00107},
}
Figure