Clinical and sensor time series go quiet for long stretches. A patient's lactate may be measured twice in an hour and then not again for a day. Every model built for such irregularly sampled data has to decide what its internal state does while nothing is observed, and that choice silently shapes its predictions at the next timestamp. Current models make this choice in one of two ways. Some hard-code a decay toward a population average, as in Temporal Belief Memory. Others let a learned black-box vector field drift wherever training pushed it, as in Neural Controlled Differential Equations. Neither approach can say, before deployment, what the model will predict after a six-hour gap that is longer than any gap seen in training.
We propose a continuous-time sequence model whose behavior during gaps is exact and can be checked. The model is a controlled differential equation (a recurrent model in continuous time whose hidden state is pushed by the incoming signal) with a ReLU vector field, driven by a causal "rectilinear" path, meaning a path that holds the last observed value flat until a new observation arrives and then jumps. With this path, every unobserved interval is governed by one fixed, autonomous, piecewise-affine ODE, which we call the drift field. A piecewise-affine ODE is one whose right-hand side is linear inside each region carved out by the ReLU hyperplanes. Because the drift field is piecewise affine, it can be solved exactly with matrix exponentials and boundary-crossing times, with no numerical solver. Its long-run behavior, including fixed points, regions that trap trajectories, and directions in which the state escapes to infinity, can be enumerated region by region. We use this to train models with a guarantee: as a gap grows, the prediction relaxes toward a calibrated prior instead of drifting into the high-confidence outer regions that make ReLU models overconfident.
Core Claim
Write the observations as $x_k \in \mathbb{R}^c$ at times $t_k$. The latent state $z \in \mathbb{R}^{n}$ evolves as
$$\frac{dz}{dt} = \sum_{i=0}^{c} \dot X_i(t)\,\big(A_i z + b_i + U_i\,\mathrm{ReLU}(Wz + w)\big),$$
where $X_0(t)=t$ is a time channel, $X_{1..c}$ are the value channels of the rectilinear path, and $A_i, U_i, b_i, W, w$ are learned. The hyperplanes $Wz+w=0$ are shared across channels, so the partition of latent space into regions does not depend on the input. Only the affine map inside each region changes with the input.
On a rectilinear path, the time segments and the value jumps are separated. During a gap of length $\Delta t$ only $X_0$ moves, so the state follows the drift field $g_0(z) = A_0 z + b_0 + U_0\,\mathrm{ReLU}(Wz+w)$ for exactly $\Delta t$ time units. At an observation, the state is integrated through the value increments while time is frozen. Both phases are autonomous piecewise-affine flows. They can therefore be solved with the switching-time machinery of Continuous-Time Piecewise-Linear Recurrent Neural Networks (cPLRNN), which solves each region in closed form, finds the exact time at which a trajectory hits a ReLU boundary, and differentiates through that time with the implicit function theorem.
The new claim has two parts. First, all effects of irregular sampling in this model pass through the single function $g_0$. Second, $g_0$ can be audited and constrained exactly, so the model's prediction after an arbitrarily long silence is a property we can certify rather than one we merely observe.
Relation to Prior Work
The closest work is cPLRNN, which supplies our solver. It is an autonomous model for reconstructing dynamical systems from sparse data. It is not driven by inputs, it is not used for classification or prediction from multichannel clinical streams, and it does not ask what happens to predictive confidence over gaps. We reuse its exact propagation and its fixed-point analysis, and we change both the object of study and the training objective.
Neural CDEs supply the controlled-equation formulation and the observation-count channel that preserves the information carried by when things are measured. Kidger et al. note that their cubic-spline paths are non-causal and point to rectilinear paths as the online alternative. We take that alternative seriously and show that, combined with a ReLU field, it makes the whole model exactly piecewise affine.
The motivating negative result is Know your limits. It proves that along rays into the outer, partially unbounded regions of a ReLU network, standard uncertainty measures converge to constants, which leaves confident "rays" extending into unseen space. That result concerns static inputs. A latent state integrated over a long gap is exactly the kind of trajectory that can travel out along such a ray, and no one has measured whether it does.
Characterizing the Discrete Geometry of ReLU Networks constrains the audit's cost. It shows that each region has on average at most $2n$ facet neighbors, so tracing one trajectory touches few candidate boundaries per step. It also reports that trained networks concentrate their regions near the data, which suggests that the outer regions we must audit are comparatively coarse.
Continuous Recurrent Units (CRU) provides the main competing principle. It uses linear latent dynamics with Kalman filtering, so its uncertainty grows with gap length in closed form, at the cost of nonlinear expressiveness. ContiFormer and decay-based recurrent networks handle gaps without any analyzable long-gap behavior.
Research Program
The first component is the model and its exact solver. We use one-hidden-layer ReLU fields, the depth regime in which cPLRNN's mathematics is established, with latent sizes from 16 to 64 and hidden widths from 64 to 256. We will report how many boundary crossings occur per gap and per observation as the width grows. This directly measures the scaling limitation that cPLRNN flags.
The second component is a diagnostic study, run before any new training objective. We take trained ReLU-field CDEs with no constraints, stretch gaps synthetically at test time, and record two things: which regions the latent trajectory visits, and whether it settles at a bounded fixed point or escapes through an unbounded region. Alongside, we record the readout's confidence as a function of gap length. If unconstrained models already relax sensibly, the guarantee machinery is less necessary and we will say so.
The third component is the guarantee. Within an unbounded region $r$, the drift is affine, $\dot z = A_r z + b_r$. A trajectory can escape to infinity inside that region only if the region's recession cone contains a direction along which $A_r$ does not contract. This can be checked exactly per region with a small linear or eigenvalue test.
We study two ways to enforce good long-gap behavior. The hard variant constrains the drift to be globally contracting toward a learned anchor $z^*$: we bound the one-sided Lipschitz constant by requiring $\mu(A_0) + \lVert U_0\rVert\,\lVert W\rVert < 0$, where $\mu$ is the logarithmic matrix norm. Under this constraint, distance to $z^*$ shrinks at least exponentially in gap length, and we tie the readout at $z^*$ to the empirical class prior. The soft variant penalizes only the unbounded regions that fail the escape test, found by sampling directions at large radius. This allows nontrivial bounded long-gap behavior, such as oscillation, while still forbidding escape. After training, an exact audit enumerates the unbounded regions where feasible and certifies a gap length beyond which predicted confidence stays below a chosen threshold.
Baselines
The naive reference point is forward-filling onto a uniform grid followed by a GRU. He et al. document that this erases the information carried by measurement timing, but it is the default clinical pipeline and anchors every metric.
GRU-D, the decay-gated recurrent network, represents hand-designed reversion to the mean. Neural CDEs report 91.9% accuracy for GRU-D on CharacterTrajectories with 70% of points dropped, against 95.3% for ODE-RNN. We must match the accuracy of the stronger models while giving decay-like guarantees.
The strongest accuracy competitor is the Neural CDE with cubic splines. It reports 98.6% accuracy on CharacterTrajectories at 70% missingness and an AUC of 0.880 on PhysioNet Sepsis. Because that spline is non-causal, we also run a Neural CDE with a rectilinear path and a smooth (tanh) field, integrated with RK4 or Dopri5. This is the fair causal comparison.
CRU is the strongest competitor on uncertainty during gaps. The file contains no reported numbers for it beyond its claims of lower interpolation error than Latent ODEs on PhysioNet and USHCN. ContiFormer is included for breadth on UEA, where it reports a best average rank of 1.9 to 2.4.
All models share data splits, a hidden-state budget matched within 10% of parameter count, and the same hyperparameter search budget of 30 trials on validation data. Each is run with five seeds.
Two ablations isolate our ingredients. The first is our model with the drift constraint removed, which isolates the guarantee. The second is a tanh-field CDE trained with the same contraction penalty and solved numerically, which isolates what exact piecewise-affine solving adds beyond the constraint itself.
Beating the baselines means three things together. Accuracy or AUC must fall within one point of the spline Neural CDE: the 0.880 Sepsis figure, and 98.6% at 70% missingness. Calibration error under stretched gaps must be lower than both CRU and the unconstrained model. Wall-clock training time must be lower than the RK4 rectilinear CDE.
Why not just…?
Why not just use a decay gate like GRU-D or Temporal Belief Memory? Decay toward a mean is exactly the long-gap behavior our hard variant certifies, so the question is fair. Decay gates impose that behavior by hand on a discrete update, and the accuracy gap reported by Kidger et al. (91.9% versus 98.6%) suggests this costs expressiveness under heavy missingness. Our model keeps a learned nonlinear flow during short gaps and reaches its prior only as gaps grow. If GRU-D matches our calibration under stretched gaps and comes within a point on accuracy, the contribution narrows to exact solving and interpretability, and we will report it that way.
Why not just train a smooth Neural CDE on a rectilinear path with a contraction penalty? This is the real threat. Contraction theory does not need ReLU, so the hard guarantee alone could be reproduced with tanh and a numerical solver. The tanh ablation settles the question on three axes: training speed, where cPLRNN reports several-fold speedups over RK4 and Tsit5 solvers; behavior on hard thresholds, where the LIF benchmark makes smooth solvers smear jumps; and the soft variant. The soft variant, which certifies "no escape" without demanding global contraction, has no exact counterpart for smooth fields. If the tanh ablation ties on all three, piecewise linearity is not earning its place.
Why not just use CRU, whose Kalman covariance already grows with the gap? CRU does provide gap-aware uncertainty in closed form, and it may calibrate well in our stress test. Its latent dynamics are linear with neural mixing of basis matrices, and f-CRU further restricts these to commuting symmetric matrices. That limits rotations and threshold behavior. We therefore expect CRU to lose on LIF-style discontinuities and on Lorenz reconstruction, but possibly to win on calibration for smooth clinical data. The PhysioNet gap-stretch experiment decides this directly.
Why not just use deep ensembles or MC dropout? Ulmer and Cinà prove that the standard uncertainty measures of ReLU models flatline along unbounded rays. An ensemble of state trajectories that escape in different directions might still disagree, so we include a five-member ensemble of the unconstrained model in the stress test rather than assume it fails.
Proposed Experiments
Accuracy and speed are measured on CharacterTrajectories and the UEA archive at 30%, 50%, and 70% dropout, and on PhysioNet 2012 mortality and PhysioNet Sepsis. Mechanism is tested on the LIF neuron system at 10% sampling density and on Lorenz-63, where cPLRNN reports qualitative success but the file has no reported number.
The decisive check is a gap-stress protocol on PhysioNet and CharacterTrajectories. At test time we delete all observations after a random cutoff, query predictions at gaps from 1× to 50× the longest training gap, and plot confidence and calibration error against gap length. We count the fraction of trajectories that escape through unbounded regions in unconstrained models, and we check that the audited models' certified gap thresholds hold empirically on held-out patients.
The proposal succeeds if three things hold together: unconstrained ReLU-field models visibly escape and stay overconfident; the certified models revert to the prior at the predicted rate; and accuracy at ordinary gaps stays within one point of the spline Neural CDE.
References
- Temporal Belief Memory: Imputing Missing Data during RNN Training
- Neural Controlled Differential Equations for Irregular Time Series
- Continuous-Time Piecewise-Linear Recurrent Neural Networks
- Know your limits: Uncertainty estimation with ReLU classifiers fails at reliable OOD detection
- Characterizing the Discrete Geometry of ReLU Networks
- Modeling Irregular Time Series with Continuous Recurrent Units
- ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling
- Early Sepsis Prediction Using Ensemble Learning with Features Extracted from LSTM Recurrent Neural Network