When we model complex physical or biological systems, we often simplify them by focusing only on a few primary variables and ignoring the rest. For instance, in molecular dynamics, we might track a large protein while ignoring the individual water molecules surrounding it.
However, this simplification comes with a steep mathematical price. The ignored degrees of freedom continue to exert an influence, acting as an environmental reservoir that retains a memory of the system's past states. This phenomenon is formally captured by frameworks like the Mori-Zwanzig formalism or the Generalized Langevin Equation (GLE), as discussed by Eric J. Parish et al. in A dynamic subgrid scale model for Large Eddy Simulations based on the Mori-Zwanzig formalism.
Instead of clean, local-in-time differential equations, we are left with non-Markovian systems containing history-dependent time-convolution integrals:
$$ \frac{dx}{dt} = f(x) + \int_0^t K(t-s) x(s) \, ds $$
To simulate this directly, you must store the entire history of $x(s)$ from $0$ to $t$. As the simulation progresses, computing this integral at every step becomes increasingly expensive, scaling quadratically ($O(t^2)$) with the number of time steps. This is the "curse of memory."
Markovian embedding is a elegant strategy to escape this computational bottleneck. Instead of looking backward in time to evaluate history integrals, we expand our perspective upward into a higher-dimensional state space. By introducing a finite set of hidden, auxiliary variables that track the state of the environment, we can trade memory for dimensionality, restoring a local, memoryless (Markovian) description.
The Core Intuition: The Exponential Trick
To understand why Markovian embedding works, consider a system where the memory kernel decays exponentially over time: $K(\tau) = e^{-\gamma \tau}$. The history-dependent term in our equation of motion can be written as an explicit variable, $z(t)$:
$$ z(t) = \int_0^t e^{-\gamma(t-s)} x(s) \, ds $$
If we want to know the value of $z(t+\Delta t)$, do we really need to re-integrate over the entire history from $0$ to $t$? Let's take the time derivative of $z(t)$ using Leibniz's rule:
$$ \frac{dz}{dt} = x(t) - \gamma \int_0^t e^{-\gamma(t-s)} x(s) \, ds = x(t) - \gamma z(t) $$
The history integral has vanished. It is replaced by a local ordinary differential equation where the rate of change of $z(t)$ depends only on the current state of the system $x(t)$ and the current value of $z(t)$ itself.
By appending $z(t)$ to our state vector, the extended system $[x(t), z(t)]$ becomes entirely Markovian. We no longer need to store or integrate over the system's history; we simply propagate $z(t)$ forward step-by-step alongside $x(t)$. This simple transformation collapses the computational complexity from quadratic ($O(t^2)$) to linear ($O(t)$).
Approximating Complex Kernels
Real-world memory kernels are rarely a single exponential decay. However, the same underlying logic applies if a complex kernel can be approximated as a finite sum of exponentials (a Prony series) or as rational functions in the Laplace domain.
As demonstrated by John Harlim et al. in Parametric reduced models for the nonlinear Schrödinger equation and Francesca Grogan et al. in Data-driven molecular modeling with the generalized Langevin equation, mapping a history-dependent kernel to rational functions allows researchers to systematically construct a set of auxiliary differential equations. Each pole or exponential term in the approximation corresponds to an additional auxiliary variable in the extended state space.
Non-Markovian System (History Integral)
│ Complexity: O(t²)
▼
Rational / Exponential Approximation of Kernel K(τ)
│
▼
Extended State Space (System + Auxiliary Variables)
│ Complexity: O(t)
▼
Markovian System (Local Differential Equations)
This quasi-Markovian approximation allows complex, non-local integro-differential equations to be solved using standard, highly optimized numerical differential equation solvers, as established early on by Jean‐Pierre Eckmann et al. in Non-Equilibrium Statistical Mechanics of Anharmonic Chains Coupled to Two Heat Baths at Different Temperatures.
Applications Across Disciplines
Because memory is a universal artifact of reducing a system's dimensions, Markovian embedding is utilized across a wide variety of scientific domains.
Molecular Dynamics and Thermostats
Standard molecular simulations often use white noise to maintain temperature, assuming an instantaneous response from the environment. However, real molecular environments exhibit correlated ("colored") noise.
In Langevin Equation with Colored Noise for Constant-Temperature Molecular Dynamics Simulations, the authors showed that by expanding the classical state vector with hidden momenta, they could map complex, memory-dependent colored noise onto a higher-dimensional Markovian framework. This technique was further expanded in Nuclear Quantum Effects in Solids Using a Colored-Noise Thermostat to efficiently capture frequency-dependent quantum corrections (like zero-point energy) without the massive overhead of path-integral simulations.
Similarly, Giovanni Bussi et al. demonstrated in Equilibrium Free Energies from Nonequilibrium Metadynamics that history-dependent metadynamics can be rigorously analyzed as a Markovian process by treating the cumulative history of the trajectory as an evolving auxiliary field.
Open Quantum Systems and Boundaries
When simulating open quantum systems or wave equations in finite domains, waves traveling outward can reflect off artificial boundaries, ruining the simulation. To prevent this, researchers use absorbing boundary conditions, which naturally introduce non-local temporal memory effects.
In Absorbing boundary conditions for time-dependent Schrödinger equations: A density-matrix formulation, the author replaced global memory kernels with local-in-time auxiliary variables, allowing for stable initial value problem formulations.
This embedding strategy is also crucial for quantum computing algorithms. In Succinct Description and Efficient Simulation of Non-Markovian Open Quantum Systems, the authors designed a quantum algorithm that maps a system with memory into a larger, memoryless Hilbert space using auxiliary proxy variables, achieving nearly optimal linear time scaling ($O(t)$) for open quantum system simulations.
Viscoelasticity and Continuum Mechanics
In materials science, the macroscopic stress of a viscoelastic material depends on its entire deformation history. Rather than tracking this history explicitly, Kaushik Bhattacharya et al. demonstrated in Learning Markovian Homogenized Models in Viscoelasticity that this memory can be captured by a finite set of internal variables. They leveraged Recurrent Neural Networks (RNNs) to learn these hidden Markovian state-space representations directly from data, creating highly efficient macroscopic surrogates.
Biological Processes
Memory also emerges in biological networks when the waiting times for molecular transitions do not follow standard exponential distributions. In Mathematical modeling of semi-Markov biomolecular processes and the memory effect, the authors showed that these semi-Markov processes can be transformed into equivalent Markovian models by deriving "effective transition rates" that dynamically incorporate historical state information based on the time elapsed since the last event.
The Fundamental Trade-off
While Markovian embedding provides an elegant escape from the quadratic scaling of history integrals, it is not a free lunch. The core challenge lies in the accuracy and efficiency of the kernel approximation.
As noted by Eric Joseph Hall et al. in Uncertainty quantification for generalized Langevin dynamics, fitting exponential or rational approximations to high-dimensional, noisy experimental data remains a complex optimization problem. If a memory kernel is highly non-trivial, it may require a massive number of auxiliary variables to achieve an acceptable fit.
If the number of auxiliary variables grows too large, the curse of dimensionality in the expanded state space can eventually outweigh the computational savings of eliminating the history integral. The art of Markovian embedding lies in finding the sweet spot: a compact set of hidden variables that sufficiently mimics the environment's memory, unlocking linear-time simulations of complex, history-dependent systems.
Key Foundational Work
Non-Equilibrium Statistical Mechanics of Anharmonic Chains Coupled to Two Heat Baths at Different Temperatures (1999) — contributed 1 concept elements to this direction.
Equilibrium Free Energies from Nonequilibrium Metadynamics (2006) — contributed 1 concept elements to this direction.