Continuous-Time Neural Models for Non-Markovian Dynamics

33 concepts32 papers1 proposal

Imagine tracking a dynamical system over time. In standard deep learning frameworks like Neural Ordinary Differential Equations (Neural ODEs) or classical Recurrent Neural Networks (RNNs), we model the system's rate of change as a function of its current state:

$$\frac{dx}{dt} = f(x(t), t)$$

This formulation relies entirely on the Markovian assumption—the principle that the future state of a system depends solely on its present state, independent of its historical trajectory. While mathematically convenient, this memoryless property fails across a vast spectrum of real-world phenomena. As [Samuel Holt] et al. point out in their work on Neural Laplace, standard continuous-time models struggle when future behavior is fundamentally shaped by integrated historical data or explicit time delays.

When we force a non-Markovian process into a Markovian framework, we encounter a severe representation bottleneck. Fernando J. Pineda noted early on in Time Dependent Adaptive Neural Networks that models relying on instantaneous steady states struggle to maintain functional memory over time without explicit transition-learning mechanisms. To capture history, a purely Markovian model must compress all past information into its current hidden state vector. If the history is rich, long-range, or contains complex propagation delays, this state vector must become unsustainably large, leading to representation collapse or computational intractability.

To break this bottleneck, a growing body of research explicitly incorporates historical trajectories directly into continuous-time neural architectures.

Discrete History via Delay Differential Networks

The most direct way to break the memoryless constraint is to allow the system's derivative to look back at specific, discrete points in the past. This shifts the framework from Ordinary Differential Equations to Delay Differential Equations (DDEs):

$$\frac{dx}{dt} = f(x(t), x(t-\tau))$$

[Srinivas Anumasa] et al. introduced this concept to deep learning with Delay Differential Neural Networks, demonstrating how allowing derivatives to depend on past states enables networks to leverage information across different depths of the network flow.

This discrete lookback is crucial for physical systems characterized by propagation delays. For instance, in air quality forecasting, pollutants emitted in one region take a physical amount of time to travel to another. [Binqing Wu] et al. leveraged this mechanism in AirDDE to explicitly model the time lag of pollutant transport. Similarly, [N. Benjamin Erichson] et al. demonstrated in Gated Recurrent Neural Networks with Weighted Time-Delay Feedback that incorporating historical time lags significantly improves an RNN's ability to capture long-term dependencies in natural processes like climate or biological cycles, where future behavior is fundamentally shaped by specific historical lags.

Continuous Memory via Integro-Differential and Fractional Frameworks

In many systems, memory is not just a sharp echo from exactly $\tau$ seconds ago; it is a continuous, cumulative footprint of the entire past trajectory. To capture this, researchers utilize Integro-Differential Equations (IDEs), which integrate over the system's history using a memory kernel $K(t, s)$:

$$\frac{dx}{dt} = f(x(t)) + \int_0^t K(t, s)x(s)ds$$

The authors of Neural Integro-Differential Equations, [Emanuele Zappala] et al., utilized this framework to model memory-heavy systems like brain activity. By combining neural networks with integral operators, they showed that tracking the cumulative history of past states yields far more accurate and interpretable models of complex neural dynamics than local, memoryless derivatives.

Another elegant way to model continuous memory is through fractional calculus, where derivatives have non-integer orders. Fractional derivatives inherently possess a "long-memory" property because their mathematical definition involves a convolution kernel that spans the entire history of the state. [Lifang Feng] et al. applied this in The memory-dependent FPK equation for fractional Gaussian noise to capture long-range correlations in complex stochastic environments like mechanical vibrations or wind loads.

However, static historical weightings in traditional fractional calculus can be rigid. To make continuous memory adaptive, [Qiyu Kang] et al. proposed Neural Fractional Attention Differential Equations (FADE). FADE replaces static kernels with learnable attention mechanisms, allowing the model to dynamically and contextually decide which historical states are relevant to the current prediction.

Bypassing Time via the Laplace Domain

While adding history terms makes models highly expressive, simulating and training them in the time domain is notoriously difficult. Tracking and integrating over past trajectories at every forward step introduces a massive computational burden.

To alleviate this, [Qingchao Jiang] et al. developed a method detailed in Neural network aided approximation and parameter inference of non-Markovian models of gene expression that uses neural networks to approximate complex non-Markovian systems with simpler, time-inhomogeneous Markovian models, easing simulation and parameter inference.

A more radical and mathematically elegant solution is to bypass the time domain entirely. [Samuel Holt] et al. developed Neural Laplace, a framework that operates in the Laplace domain. In the Laplace domain, complex history-dependent operations like convolutions, integrals, and time delays map to simple algebraic operations. By training a neural network to predict the system's representation in this frequency-like space, the model can simulate non-Markovian dynamics—including delay and integro-differential equations—without explicitly stepping through time or tracking history step-by-step.

Why Non-Markovian Modeling Matters Across Disciplines

Breaking the memoryless assumption is proving essential across diverse scientific fields where history cannot be compressed into an instantaneous state vector:

Key Foundational Work

Time Dependent Adaptive Neural Networks (1989) — contributed 1 concept elements to this direction.
Predictive Representations of State (2001) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_memoryless_state_transitions_9ac25acb,
  title = {memoryless state transitions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/memoryless-state-transitions/rdr_9ac25acb28ca4fc4988fa33c568d08e7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_memoryless_state_transitions_9ac25acb_v1,
  title = {memoryless state transitions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/memoryless-state-transitions/rdr_9ac25acb28ca4fc4988fa33c568d08e7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake