Mean first exit times of Ornstein–Uhlenbeck processes in high-dimensional spaces

Hans Kersting, Antonio Orvieto, Frank Proske, Aurélien LucchiJournal of Physics A Mathematical and Theoretical 2022

Abstract

The $d$-dimensional Ornstein--Uhlenbeck process (OUP) describes the trajectory of a particle in a $d$-dimensional, spherically symmetric, quadratic potential. The OUP is composed of a drift term weighted by a constant $θ\geq 0$ and a diffusion coefficient weighted by $σ> 0$. In the absence of drift (i.e. $θ= 0$), the OUP simply becomes a standard Brownian motion (BM). This paper is concerned with estimating the mean first-exit time (MFET) of the OUP from a ball of finite radius $L$ for large $d \gg 0$. We prove that, asymptotically for $d \to \infty$, the OUP takes (on average) no longer to exit than BM. In other words, the mean-reverting drift of the OUP (scaled by $θ\geq 0$) has asymptotically no effect on its MFET. This finding might be surprising because, for small $d \in \mathbb{N}$, the OUP exit time is significantly larger than BM by a margin that depends on $θ$. As it allows for the drift to be ignored, it might simplify the analysis of high-dimensional exit-time problems in numerous areas. Finally, our short proof for the non-asymptotic MFET of OUP, using the Andronov--Vitt--Pontryagin formula, might be of independent interest.

Introduction

The Ornstein–Uhlenbeck process (OUP) is a cornerstone of stochastic modeling, widely applied across physics, biology, and finance. Unlike standard Brownian motion (BM), which diffuses freely, the OUP incorporates a mean-reverting drift that pulls the process toward a central equilibrium. In low-dimensional settings, this inward pull significantly delays the time it takes for a particle to exit a bounded region compared to a drift-less BM.

However, modern scientific challenges—from modeling large-scale neural networks to analyzing the escape of Stochastic Gradient Descent (SGD) from local minima in deep learning—often involve high-dimensional spaces. This research investigates whether the intuitive delay caused by OUP drift persists as the dimensionality $d$ grows.

Methodology

The study employs a rigorous analytical framework to derive the mean first-exit time (MFET):

  1. Radial Reduction: Exploiting the spherical symmetry of the OUP, the $d$-dimensional process is reduced to a one-dimensional radial OUP (rOUP), tracking only the distance $\rho$ from the origin.
  2. AVP Formula: The Andronov–Vitt–Pontryagin (AVP) formula is used to convert the stochastic exit-time problem into a deterministic Boundary Value Problem (BVP).
  3. Analytical Bounds: The MFET is reformulated using upper incomplete Gamma functions. By applying sharp analytical bounds, the authors characterize the behavior of the exit time in the limit as $d \to \infty$.
  4. Numerical Validation: Theoretical results are corroborated through Monte Carlo simulations of OUP and BM trajectories across dimensions ranging from $d=2$ to $d=10,000$.

The Dominance of Diffusion in High Dimensions

The central finding of this work is that in high-dimensional regimes, the "outward pressure" of diffusion overwhelms the "inward pull" of the mean-reverting drift. In the radial stochastic differential equation, the term representing the outward push due to dimensionality ($\sigma^2 d$) grows linearly with $d$. Conversely, the inward drift term ($2\theta \rho^2$) remains bounded because the particle is confined within a ball of fixed radius $L$.

Comparison of OUP and BM trajectories and radial distances for $d=2, 10, 1000$. As dimensionality increases, the radial behavior of the OUP (cyan) converges to that of Brownian motion (blue).

As $d$ increases, the OUP behaves increasingly like a squared Bessel process (the radial counterpart of BM). This suggests that for sufficiently high-dimensional problems, the complex drift terms can be effectively ignored in exit-time calculations.

Key Results

  • Exact MFET Representation: The paper derives a new, concise formula for the MFET of an OUP in terms of the upper incomplete Gamma function.
  • Asymptotic Equivalence: The core result demonstrates that for $d \to \infty$, the MFET scales as: $$\mathbb{E}^x[\tau_L] \sim \frac{L^2 - x^2}{\sigma^2} d^{-1}$$ This is identical to the scaling of standard Brownian motion, proving that the drift coefficient $\theta$ has no leading-order impact in high dimensions.

  • Convergence Rate: While OUP exit times are strictly greater than BM exit times for any finite $d$, their ratio approaches unity as $d$ grows, with both exit times shrinking at a rate of $1/d$.

Discussion and Future Directions

This "drift-erasing" effect is contingent on the exit boundary $L$ remaining fixed. If the boundary were to scale with the dimensionality (e.g., $L \propto \sqrt{d}$), the drift would retain its influence. Furthermore, while this study assumes a spherically symmetric quadratic potential, future research could explore non-symmetric potentials characteristic of the complex loss landscapes in machine learning.

These findings provide a theoretical foundation for simplifying high-dimensional stochastic analysis, potentially offering new insights into how noise facilitates the escape of optimization algorithms from local minima in deep learning models.

Summary updated

Abstract

Abstract The d -dimensional Ornstein–Uhlenbeck process (OUP) describes the trajectory of a particle in a d -dimensional, spherically symmetric, quadratic potential. The OUP is composed of a drift term weighted by a constant <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mi>θ</mml:mi> <mml:mo>⩾</mml:mo> <mml:mn>0</mml:mn> </mml:math> and a diffusion coefficient weighted by σ &gt; 0. In the absence of drift (i.e. θ = 0), the OUP simply becomes a standard Brownian motion (BM). This paper is concerned with estimating the mean first-exit time (MFET) of the OUP from a ball of finite radius L for large <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mi>d</mml:mi> <mml:mo>≫</mml:mo> <mml:mn>0</mml:mn> </mml:math> . We prove that, asymptotically for <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mi>d</mml:mi> <mml:mo stretchy="false">→</mml:mo> <mml:mi mathvariant="normal">∞</mml:mi> </mml:math> , the OUP takes (on average) no longer to exit than BM. In other words, the mean-reverting drift of the OUP (scaled by <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mi>θ</mml:mi> <mml:mo>⩾</mml:mo> <mml:mn>0</mml:mn> </mml:math> ) has asymptotically no effect on its MFET. This finding might be surprising because, for small <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mi>d</mml:mi> <mml:mo>∈</mml:mo> <mml:mrow> <mml:mi mathvariant="double-struck">N</mml:mi> </mml:mrow> </mml:math> , the OUP exit time is significantly larger than BM by a margin that depends on θ . As it allows for the drift to be ignored, it might simplify the analysis of high-dimensional exit-time problems in numerous areas. Finally, our short proof for the non-asymptotic MFET of OUP, using the Andronov–Vitt–Pontryagin formula, might be of independent interest.
BibTeX
@article{kersting2022mean,
  author = {Hans Kersting and Antonio Orvieto and Frank Proske and Aurélien Lucchi},
  title = {{Mean first exit times of Ornstein–Uhlenbeck processes in high-dimensional spaces}},
  journal = {Journal of Physics A Mathematical and Theoretical},
  year = {2022},
}

Loading related content...

Figure