When sensors drop out, the usual approach to learning a dynamical model is to fill the gaps first and fit the model second. That order is backwards, because good gap-filling requires the model. A cleaner route exists for linear systems. One can search the recorded data directly for "annihilators": fixed linear combinations of a few consecutive measurements that always equal zero, because the physics forces them to. Once these relations are found, the missing values follow from simple algebra. This proposal extends that idea to nonlinear systems. It also argues that the property of a relation that matters most under missing data is how few measurements it touches. A relation spanning three entries can be learned from every stretch of data where those three entries happen to be observed. A relation spanning thirty entries needs thirty consecutive, fully observed entries, and under realistic dropout such stretches almost never occur.
The core claim, in two sentences: lift the raw measurements through a dictionary of nonlinear functions, and then look for the sparsest zero-valued relations, each estimated from its own set of partially complete windows. In that setting, feature redundancy stops being a nuisance. Redundant sensors and short-lag couplings supply short relations, and short relations remain learnable under heavy missingness, so the method should tolerate dropout rates at which full-window identification and regression both run out of usable data.
What exists and what is missing
The direct ancestor is Data-based system representations from irregularly measured data. That paper finds complete sub-blocks inside a Hankel matrix with missing entries, where a Hankel matrix is the matrix whose columns are sliding windows of the time series. It extracts the left null space of those sub-blocks, meaning the annihilators, and stacks short annihilators across time shifts to represent arbitrary horizons. Missing data is then filled in closed form. It identified dynamics with up to 30% missing data and ran in 0.09 s, against 57.9 s for a nuclear-norm completion heuristic. We reuse its method: kernel retrieval and shift-chain stacking. Its limits define our gap. It is restricted to linear time-invariant systems, it needs the system order in advance, it has recovery guarantees only for periodic missingness, and it returns point completions with no uncertainty.
The "obvious" nonlinear extension replaces raw measurements with lifted features, as in Koopman-style methods, and runs the same algorithm. We treat this as our key ablation rather than our method, for two reasons. First, a nonlinear dictionary is massively redundant. Identities such as $\sin^2+\cos^2=1$ and collinear physical sensors create null-space directions that hold for any signal. These directions carry no information about the dynamics, and a naive rank test cannot distinguish them from real dynamical laws. Second, a lifted window is complete only if every raw entry it depends on is observed. Full-window methods therefore lose data very quickly as missingness grows.
Two works show that redundancy and missingness interact. On-line reconstruction of missing data in sensor/actuator networks by exploiting temporal and spatial redundancy reconstructs dropped channels from lagged neighbouring sensors, pruning inputs with a cross-correlation dependency graph. It reached a 90.81% fit on the Mount St. Martino rock-face network using only a pruned subset of sensors. The paper also reports that linear models leave non-white residuals on nonlinear data, which motivates us. Its reconstructions, however, are deterministic, and its pruning is linear. A Sparse Bayesian Deep Learning Approach for Identification of Cascaded Tanks Benchmark shows that sparsity priors plus uncertainty estimates help nonlinear identification from little data, but it assumes uninterrupted measurements. On the theory side, Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples and Physics-informed learning under mixing: How physical knowledge speeds up learning both find that temporal dependence mainly lengthens a burn-in period rather than slowing the eventual learning rate. Both assume continuous observation streams, so they tell us which question to ask but cannot answer it for gappy data.
Core Claim
Let $\phi$ be a dictionary mapping raw measurements to lifted features. An annihilator is a coefficient vector $c$ supported on a set $S$ of (feature, lag) pairs such that $\sum_{(k,\ell)\in S} c_{k\ell}\,\phi_k(z_{t+\ell}) = 0$ for all $t$, where $z_t$ is the raw measurement vector at time $t$. Suppose each raw entry is missing independently with probability $p$, and $S$ depends on $m(S)$ raw entries. Then the number of time indices at which $c$ can be checked against data is roughly
$$n_S \approx N(1-p)^{m(S)},$$
where $N$ is the series length. The expression holds because each of the $m(S)$ entries must be present. Full-window methods pay $m = $ (number of channels) × (window length). Sparse relations pay far less.
Our claim is that estimating each candidate relation from its own $n_S$ windows, and completing gaps with the sparsest relation available at each gap, is the right way to combine the four directions in this problem. Nonlinearity enters through the dictionary. Redundancy supplies short relations. Missingness sets the price of long ones. Temporal dependence determines how much information $n_S$ correlated windows actually contain.
Research Program
The algorithm works in three stages.
The first stage is a support-aware kernel search. We grow candidate supports greedily, in the spirit of orthogonal matching pursuit, and start with single-time-slice supports. These catch static redundancies such as collinear sensors and dictionary identities. For each candidate support, we assemble the sub-Hankel matrix from exactly the windows complete on that support and test its smallest singular value against a threshold. The threshold is computed from a concentration bound for dependent data rather than from a fixed tolerance. Sparse Regression with $\ell_0$ Constraints for $\alpha$-Mixing Time Series: Algorithms and Guarantees supplies the template for support selection under strong mixing. An Empirical Bernstein Inequality for Dependent Data in Hilbert Spaces and Applications supplies a data-driven variance term.
Signal-independent relations are identified and removed first. A relation is signal-independent if it vanishes on random inputs pushed through the dictionary. Removing these relations projects the lifted coordinates onto a smaller, non-redundant space. Only then do we search for dynamical relations. This order is what keeps dictionary redundancy from contaminating the kernel.
The second stage is gap completion. For each missing entry, we choose the sparsest discovered relation whose other entries are observed or have already been completed. We then solve the resulting small one-variable nonlinear equation, which is exact recursion whenever the entry appears linearly. Shift-chain stacking, as in Alsalti et al., carries completion across long gaps.
The third stage is uncertainty. We refit the kernel on moving-block bootstrap resamples of the usable windows. The spread of completions across resamples, combined with each relation's residual variance, gives a predictive interval for every filled entry. This is a deliberately cheap substitute for full MCMC, which the file flags as intractable at scale. Its role is to keep downstream fits from treating imputed values as certain.
The theory task is to prove a finite-sample guarantee for kernel recovery on support $S$ under β-mixing observations and independent missingness. The target result is that the recovery error is governed by an effective count of order $n_S/\tau_{\text{mix}}$, where $\tau_{\text{mix}}$ is the mixing time. This result would settle, for this estimator, whether missingness only lengthens burn-in or also slows the rate, which is the question the file leaves open. We will also characterize, as a negative result if needed, how burst or state-triggered missingness breaks the argument.
The scope is honest. The method is exact only for systems whose laws are linear in a finite dictionary, such as polynomial, trigonometric, or square-root laws. It will degrade on non-smooth effects such as tank overflow.
Baselines
The naive reference is linear interpolation of gaps followed by a sparse polynomial NARX model (nonlinear autoregressive model with exogenous inputs), fit by least squares with the same dictionary we use. It shows what the problem costs if one ignores it.
The strongest direct competitor on the identification question is sparse regression on windows that are complete across all channels and lags, using the $\ell_0$ method for mixing series cited above. It shares our dictionary and sparsity prior and differs only in discarding every window with any gap. We will beat it if our method gives lower held-out simulation error at equal or higher missingness, and if the advantage grows with $p$ in the way $(1-p)^{m}$ predicts.
The key ablation is Alsalti et al.'s kernel retrieval applied to the lifted dictionary without the support-aware search and without removing static redundancy. This is the obvious idea. It isolates whether sparse, per-support estimation is the ingredient that matters. A second ablation removes only the static-redundancy stage.
For the spatial-redundancy setting, the competitor is Alippi et al.'s dependency-graph MISO (multiple-input single-output) reconstructor, run on the same gaps. For uncertainty-aware nonlinear identification, the competitor is the sparse Bayesian network of the Cascaded Tanks paper, trained on interpolated data. On complete data that paper reports long-term simulation RMSE 0.344, against 0.777 for an unregularized network. We did not find a result on Lacuna for any method on Cascaded Tanks with missing data, so all missing-data numbers will be produced here. As a flexible black box, we include a masked recurrent network given the missingness mask as input.
Every method receives the same dictionary where applicable, the same missingness masks drawn from the same seeds, the same hyperparameter-search budget on a validation split, and the same wall-clock budget. Runtime is reported alongside accuracy.
Why not just…?
Why not just impute with interpolation and then fit a nonlinear model? This is the chicken-and-egg trap that Alsalti et al. describe: interpolation ignores the governing laws. Alippi et al. show that dynamics-agnostic reconstructions leave structured residuals on nonlinear data. The naive baseline tests this directly. We expect it to be competitive at 10% random dropout and to fail at 40% or more and under gaps longer than the system's time constants. If it does not fail there, our method has little reason to exist, and we will report that.
Why not just run sparse regression on the complete windows? This is the real threat. At low missingness it should match us, since it estimates the same relations. Our advantage exists only when the usable-window count for full windows collapses while short relations remain estimable. The decisive plot shows error versus $p$ for both methods, overlaid with the counted usable windows per relation. If the two error curves do not separate where the window counts do, the thesis is wrong.
Why not just lift the data and apply the existing incomplete-Hankel method? We expect two failures. Dictionary identities will produce spurious annihilators, and full lifted windows will be rare. The ablation measures both effects: the number of false annihilators accepted, and completion error.
Why not just train a masked sequence network? On complete Cascaded Tanks data, an unregularized network more than doubles the RMSE of the sparse Bayesian model (0.777 vs 0.344). The physics-informed mixing paper also finds slower error decay for unconstrained networks on dependent data, $T^{-0.681}$ against $T^{-1.086}$. Gappy data can only shrink the effective sample. The masked-network baseline tests whether this holds under missingness. We will also check whether its intervals, obtained from an ensemble, are calibrated.
Proposed Experiments
We start with Lotka-Volterra, the predator-prey benchmark from Probabilistic Forecasting for Dynamical Systems with Missing or Imperfect Data. We add redundant noisy copies and polynomial mixtures of the two populations to create feature redundancy. Here the true annihilators are known exactly, so we can measure how precisely and completely the relations are recovered.
Next is the Cascaded Tanks benchmark, with its 1,024 training and 1,024 test samples and a square-root dictionary. We impose missing completely at random (MCAR) dropout at 10% to 60% and burst dropout, and report long-term simulation RMSE.
The Mount St. Martino rock-face network, with 8 units, about 9,000 samples, and lagged thermal coupling, tests real sensor redundancy and delay. The metric is Alippi et al.'s fit score on artificially removed but known values, including the root-node sensor Unit 7, which no neighbour leads in time.
The multi-arm pendulum from The Experimental Multi-Arm Pendulum on a Cart: A Benchmark System for Chaos, Learning, and Control serves as a chaotic stress test with a trigonometric dictionary. That paper releases data but reports no accuracy numbers.
Metrics are completion RMSE, simulation RMSE, 90% interval coverage and width, the number of spurious annihilators accepted, and runtime.
Three decisive checks follow. The first asks whether the gap between our method and complete-window regression tracks the predicted ratio of usable windows as $p$ grows. The second asks whether removing the static-redundancy stage measurably increases spurious relations and completion error on the redundant Lotka-Volterra variant. The third applies state-triggered missingness, where a sensor drops when the state is extreme, to find where the independence assumption fails. We will report where coverage degrades rather than hiding the failure.
References
- Data-based system representations from irregularly measured data
- On-line reconstruction of missing data in sensor/actuator networks by exploiting temporal and spatial redundancy
- A Sparse Bayesian Deep Learning Approach for Identification of Cascaded Tanks Benchmark
- Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples
- Physics-informed learning under mixing: How physical knowledge speeds up learning
- Sparse Regression with $\ell_0$ Constraints for $\alpha$-Mixing Time Series: Algorithms and Guarantees
- An Empirical Bernstein Inequality for Dependent Data in Hilbert Spaces and Applications
- Probabilistic Forecasting for Dynamical Systems with Missing or Imperfect Data
- The Experimental Multi-Arm Pendulum on a Cart: A Benchmark System for Chaos, Learning, and Control