Standard deep learning models for sequences, such as Recurrent Neural Networks (RNNs) and Transformers, operate on a "ticking clock" logic. They assume that data arrives in discrete, evenly spaced intervals. However, the real world is rarely so polite. Medical records, financial transactions, and physical sensors produce "sporadic" or irregularly sampled time series where the gap between observations carries as much information as the observations themselves.
To bridge this gap, a growing body of research has converged on a hybrid architecture: Neural Jump Ordinary Differential Equations (NJODEs). This approach treats the hidden state of a model as a continuous physical process that "flows" over time but "jumps" instantaneously when new information arrives.
The Intuition: Flow and Correction
Imagine tracking the position of a drone. Between your visual sightings (observations), the drone follows the laws of physics—a continuous trajectory. When you catch a glimpse of it at a new location, you immediately update your mental model of where it is and how fast it's going.
In NJODEs, this is formalized through two distinct mechanisms:
- The Flow (Continuous Evolution): Between observations $t_i$ and $t_{i+1}$, the latent state $h(t)$ evolves according to an Ordinary Differential Equation (ODE), typically parameterized by a neural network: $\frac{dh}{dt} = f_\theta(h(t), t)$. This allows the model to represent the system's natural decay, growth, or oscillation during periods of silence.
- The Jump (Discrete Update): At the moment an observation arrives, the latent state undergoes an instantaneous shift. This is often implemented as a "jump map" or a gated update (like a GRU or LSTM cell) that incorporates the new data point to "steer" the trajectory back toward reality.
As Yulia Rubanova, David Duvenaud, and Tian Qi Chen demonstrated in Latent Ordinary Differential Equations for Irregularly-Sampled Time Series, this allows the model to naturally handle missing data. If a sensor goes dark for an hour, the ODE simply integrates over that hour, maintaining a consistent internal state.
Modeling Shocks and Interventions
The hybrid approach is particularly powerful for systems defined by "shocks"—sudden external forces that disrupt a smooth process. Junteng Jia and Austin R. Benson extended this to Neural Jump Stochastic Differential Equations (JSDEs) in Neural Jump Stochastic Differential Equations, where the jumps themselves are treated as stochastic events, like earthquakes or sudden medical crises.
This framing is essential for Intervention Modeling. In Neural Ordinary Differential Equations for Intervention Modeling, Michael Poli and Stefano Massaroli show that medical treatments can be modeled as discrete jump maps that modify the continuous latent health trajectory of a patient. By separating the "natural" disease progression (the ODE) from the "treatment effect" (the jump), the model becomes more interpretable and better at counterfactual reasoning.
Scaling to Complex Systems
While the early work focused on simple time series, the "flow-jump" paradigm has expanded into more complex domains:
- Graphs: He Yu and Jing Liu applied this to dynamic graphs in Community-Aware Temporal Walks, where node representations evolve continuously but update during discrete interactions.
- Point Processes: In Deep Continuous-Time State-Space Models for Marked Event Sequences, Yuxin Chang and Alex Boyd introduced the State-Space Point Process (S2P2). They treat events as "impulses" in a differential equation, allowing the model to scale linearly with sequence length while maintaining the continuous-time elegance of classical Hawkes processes.
- Physics: To prevent error accumulation in long-term simulations, Fan Xu and colleagues proposed CoPS in Breaking the Discretization Barrier of Continuous Physics Simulation Learning, which interleaves continuous ODE evolution with discrete "neural corrections" to stabilize the simulation.
Why It Works: The Best of Both Worlds
The success of the flow-jump architecture stems from its ability to resolve the tension between temporal flexibility and computational stability. Purely continuous models (like standard Neural ODEs) can struggle with sudden shifts in data, while purely discrete models (like RNNs) struggle with irregular timing.
By using an ODE to handle the "between-times" and a discrete update to handle the "at-times," researchers like Florian Krach and Josef Teichmann have shown in Learning Chaotic Systems and Long-Term Predictions with Neural Jump ODEs that these models can converge toward optimal statistical predictors even for chaotic or partially observed systems. The jump provides the necessary "reset" or "correction" that keeps the continuous flow aligned with the observed world.