Imagine tracking a dynamical system over time. In standard deep learning frameworks like Neural Ordinary Differential Equations (Neural ODEs) or classical Recurrent Neural Networks (RNNs), we model the system's rate of change as a function of its current state:
$$\frac{dx}{dt} = f(x(t), t)$$
This formulation relies entirely on the Markovian assumption—the principle that the future state of a system depends solely on its present state, independent of its historical trajectory. While mathematically convenient, this memoryless property fails across a vast spectrum of real-world phenomena. As [Samuel Holt] et al. point out in their work on Neural Laplace, standard continuous-time models struggle when future behavior is fundamentally shaped by integrated historical data or explicit time delays.
When we force a non-Markovian process into a Markovian framework, we encounter a severe representation bottleneck. Fernando J. Pineda noted early on in Time Dependent Adaptive Neural Networks that models relying on instantaneous steady states struggle to maintain functional memory over time without explicit transition-learning mechanisms. To capture history, a purely Markovian model must compress all past information into its current hidden state vector. If the history is rich, long-range, or contains complex propagation delays, this state vector must become unsustainably large, leading to representation collapse or computational intractability.
To break this bottleneck, a growing body of research explicitly incorporates historical trajectories directly into continuous-time neural architectures.
Discrete History via Delay Differential Networks
The most direct way to break the memoryless constraint is to allow the system's derivative to look back at specific, discrete points in the past. This shifts the framework from Ordinary Differential Equations to Delay Differential Equations (DDEs):
$$\frac{dx}{dt} = f(x(t), x(t-\tau))$$
[Srinivas Anumasa] et al. introduced this concept to deep learning with Delay Differential Neural Networks, demonstrating how allowing derivatives to depend on past states enables networks to leverage information across different depths of the network flow.
This discrete lookback is crucial for physical systems characterized by propagation delays. For instance, in air quality forecasting, pollutants emitted in one region take a physical amount of time to travel to another. [Binqing Wu] et al. leveraged this mechanism in AirDDE to explicitly model the time lag of pollutant transport. Similarly, [N. Benjamin Erichson] et al. demonstrated in Gated Recurrent Neural Networks with Weighted Time-Delay Feedback that incorporating historical time lags significantly improves an RNN's ability to capture long-term dependencies in natural processes like climate or biological cycles, where future behavior is fundamentally shaped by specific historical lags.
Continuous Memory via Integro-Differential and Fractional Frameworks
In many systems, memory is not just a sharp echo from exactly $\tau$ seconds ago; it is a continuous, cumulative footprint of the entire past trajectory. To capture this, researchers utilize Integro-Differential Equations (IDEs), which integrate over the system's history using a memory kernel $K(t, s)$:
$$\frac{dx}{dt} = f(x(t)) + \int_0^t K(t, s)x(s)ds$$
The authors of Neural Integro-Differential Equations, [Emanuele Zappala] et al., utilized this framework to model memory-heavy systems like brain activity. By combining neural networks with integral operators, they showed that tracking the cumulative history of past states yields far more accurate and interpretable models of complex neural dynamics than local, memoryless derivatives.
Another elegant way to model continuous memory is through fractional calculus, where derivatives have non-integer orders. Fractional derivatives inherently possess a "long-memory" property because their mathematical definition involves a convolution kernel that spans the entire history of the state. [Lifang Feng] et al. applied this in The memory-dependent FPK equation for fractional Gaussian noise to capture long-range correlations in complex stochastic environments like mechanical vibrations or wind loads.
However, static historical weightings in traditional fractional calculus can be rigid. To make continuous memory adaptive, [Qiyu Kang] et al. proposed Neural Fractional Attention Differential Equations (FADE). FADE replaces static kernels with learnable attention mechanisms, allowing the model to dynamically and contextually decide which historical states are relevant to the current prediction.
Bypassing Time via the Laplace Domain
While adding history terms makes models highly expressive, simulating and training them in the time domain is notoriously difficult. Tracking and integrating over past trajectories at every forward step introduces a massive computational burden.
To alleviate this, [Qingchao Jiang] et al. developed a method detailed in Neural network aided approximation and parameter inference of non-Markovian models of gene expression that uses neural networks to approximate complex non-Markovian systems with simpler, time-inhomogeneous Markovian models, easing simulation and parameter inference.
A more radical and mathematically elegant solution is to bypass the time domain entirely. [Samuel Holt] et al. developed Neural Laplace, a framework that operates in the Laplace domain. In the Laplace domain, complex history-dependent operations like convolutions, integrals, and time delays map to simple algebraic operations. By training a neural network to predict the system's representation in this frequency-like space, the model can simulate non-Markovian dynamics—including delay and integro-differential equations—without explicitly stepping through time or tracking history step-by-step.
Why Non-Markovian Modeling Matters Across Disciplines
Breaking the memoryless assumption is proving essential across diverse scientific fields where history cannot be compressed into an instantaneous state vector:
- Biology and Chemistry: Traditional models treat cellular transitions or gene expression as memoryless. However, [Philipp Weiler] et al. showed in CellRank 2 that long-term epigenetic memory heavily influences developmental trajectories. Similarly, [Mahboobeh Ghorbani] et al. demonstrated in Gene Expression Is Not Random that cross-talk within gene regulatory networks exhibits persistent, long-range fractal dependencies. Even at sub-nanosecond scales, [Fabrizio Marinelli] et al. found in A Kinetic Model of Trp-Cage Folding from Multiple Biased Molecular Dynamics Simulations that the physical memory of previous molecular configurations shapes transition states.
- Cognitive Science and Safety Validation: Standard reinforcement learning assumes a Markov decision process. Yet, [Li Ji-An] et al. discovered in Discovering cognitive strategies with tiny recurrent neural networks that human and animal cognitive behaviors are inherently non-Markovian, requiring architectures that do not compress history into a single static state. This has practical implications for safety; [Mykel Kochenderfer] et al. noted in Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decomposition that validation frameworks assuming Markovian dynamics fail to evaluate autonomous agents that rely on long-term internal memory.
- Stochastic Systems and Physics: In stochastic modeling, Erhan Bayraktar solved Stochastic Differential Games in a Non-Markovian Setting to capture long-range dependence in financial markets. When reverse-engineering physical laws from trajectory data, [Youngkyoung Bae] et al. demonstrated in Inferring the Langevin Equation with Uncertainty via Bayesian Neural Networks that accounting for memory is vital for robust uncertainty estimation. Finally, [Tanmay Rajpurohit] et al. highlighted in Stochastic Thermodynamics: A Dynamical Systems Approach that extending thermodynamic formulations to non-Markovian systems remains a critical frontier for capturing real-world memory effects.
Key Foundational Work
Time Dependent Adaptive Neural Networks (1989) — contributed 1 concept elements to this direction.
Predictive Representations of State (2001) — contributed 1 concept elements to this direction.