Many physical systems are tracked online. Examples are a building's thermal behaviour, a circuit, or a city's traffic. The tracker has to follow hidden quantities that change every step (the state) and also physical constants that change slowly (the parameters), such as a thermal resistance or a conductance. Every such tracker carries one knob that matters more than the rest: how fast the parameter estimates are allowed to move. Set it too loose and the parameters wander off during calm periods, when the data carry no information about them. Set it too tight and the tracker cannot follow a real change in the system after a storm, a holiday, or a switch in operating mode. Today this knob is a hand-picked constant, or a hand-thresholded "freeze" switch. Choosing it well takes many full re-runs of the estimator.
We propose to make the knob a small learned function of observed outside signals (weather, calendar, control inputs) and of how informative the recent data are. The function is learned in a single pass over the data. The gradient used to learn it comes from a short backward-in-time pass over the last few steps. This is the same kind of pass a fixed-lag smoother (an estimator that revises each estimate using a few steps of future data) already runs. The proposal therefore ties together four threads into one mechanism. Outside variables decide when change is plausible. A bounded look-back supplies hindsight. The tuning that used to need an outer search loop happens online.
Core Claim
Consider an estimator that treats parameters as slowly drifting quantities inside the state. Write the drift as $\theta_{t+1} = \theta_t + w_t$, where $\theta_t$ is the parameter vector at time $t$ and $w_t$ is zero-mean noise with covariance $$Q^\theta_t = \mathrm{diag}\big(\exp g_\phi(u_t, e_t)\big),$$ Here $u_t$ is the vector of outside (exogenous) signals at time $t$, and $e_t$ is a running measure of excitation, meaning how much the recent inputs and states reveal about the parameters. $g_\phi$ is a small network with weights $\phi$.
We fit $\phi$ online by gradient ascent on the estimator's one-step-ahead predictive log-likelihood. The gradient is taken through only the last $L$ steps of the filter recursion: $$\phi \leftarrow \phi + \beta\, \nabla_\phi \sum_{s=t-L+1}^{t} \log p(y_s \mid y_{1:s-1}; \phi).$$ In this update, $y_s$ is the observation at step $s$, $\beta$ is a step size, and the gradient is truncated at window $L$.
The claim has two parts. First, a drift rate gated by outside context should beat both a tuned constant and an excitation-only freeze rule. The advantage should be largest in regimes where outside shocks and quiet phases are interleaved. Second, the reverse pass through $L$ filter steps has the structure and cost of an $L$-lag backward smoothing pass. If so, the hyperparameter gradient comes nearly for free from the bidirectional computation, and its window $L$ sets a single, interpretable trade-off. A small $L$ gives the greedy, short-sighted tuning seen in local hypergradients. A large $L$ approaches the expensive full-trajectory gradient. Whether this structural correspondence holds exactly for nonlinear sigma-point filters is something we must derive and check numerically. We do not assume it.
Why This Is a Gap
The research file shows each ingredient in isolation, and the pieces do not yet meet.
Two papers motivate the problem. Nonlinear moving horizon estimation for robust state and parameter estimation estimates parameters over a sliding window of recent data (moving horizon estimation). It shows that this estimator drifts badly when excitation is lost. It fixes the drift by monitoring excitation and inflating regularization to freeze the parameters. However, the freeze thresholds are chosen before runtime, and outside inputs enter only as classical control signals. Robust online joint state/input/parameter estimation of linear systems handles outliers and unknown inputs well. It is restricted to linear dynamics, runs forward only, and its tuning constants are set by hand. The file points out the tension between them: a sudden outside event can look like rich excitation. A freeze rule driven only by excitation may then let the parameters chase an unmodelled shock. Our gate sees $u_t$ directly, so it can learn that "rain started" means "do not reinterpret this residual as a parameter change."
Other papers supply methods we reuse. Dual Estimation and the Unscented Transformation provides the sigma-point machinery for estimating nonlinear states and parameters jointly. We use an augmented-state unscented filter. Treating parameters as states collapses the feared three-level problem (states, parameters, hyperparameters) into two levels: a deterministic filter recursion inside, with $\phi$ outside. Particle Metropolis-Hastings using gradient and Hessian information tracks parameter gradients with a fixed-lag particle filter. It is the closest precedent for using a bounded look-back to obtain gradients. We differ in two ways. Our gradient targets the drift-rate hyperparameters rather than $\theta$ itself, and our drift rate is conditioned on outside context.
Several papers constrain the design. Online Learning Rate Adaptation with Hypergradient Descent shows that one-step hypergradients are cheap but introduce a secondary step size $\beta$. They also optimize greedily. We must report sensitivity to $\beta$, and we treat $L=1$ as the greedy case. Glocal Hypergradient Estimation with Koopman Operator fixes that greediness differently: it extrapolates the long-horizon gradient from a short trajectory of local gradients using dynamic mode decomposition. Its authors note that it assumes smooth, linearizable optimization dynamics, and this may fail under chaotic forcing. We therefore run it as a competitor rather than adopt it. Bidirectional Learning for Time-series Models with Hidden Units shows that combining forward and time-reversed passes regularizes latent models. It also reports that its backward model is a heuristic and that it is very sensitive to learning rates.
We avoid that heuristic reversal. Our backward pass is the standard smoothing recursion, which conditions on the outside inputs as known quantities rather than reversing them. This addresses the "does exogenous forcing run backwards?" asymmetry raised in the file's Gap 2 without inventing a reversed process. Self-Tuning Networks shows that hyperparameter schedules learned online can beat any static value: 70.30 against 72.13 for Bayesian optimization on Penn Treebank perplexity. This is the main evidence that a learned, time-varying drift rate could beat a tuned constant one.
Research Program
We will build three components. The first is a differentiable augmented-state unscented filter with known outside inputs and the context-gated drift model above. The second is a fixed-lag backward pass that returns two things: smoothed estimates of states and parameters, and the truncated predictive-likelihood gradient with respect to $\phi$. The third is an online loop that updates $\phi$ every step. Measurement noise and the state process noise are tuned the same way, as constants, so that the only new ingredient is the gated parameter drift.
The analysis has three parts. First, we will derive the correspondence between the $L$-step reverse-mode gradient and the $L$-lag smoother for the linear-Gaussian case. We will then measure numerically how far the two diverge for the unscented filter on nonlinear systems. Second, we will characterize the bias of the gradient as a function of $L$. We expect it to shrink roughly as the filter's error-forgetting rate raised to the power $L$, because a stable filter forgets the influence of early steps geometrically. Third, we will add a stability safeguard by clipping $g_\phi$ to a bounded range. We will test whether the boundedness argument of the excitation-monitored estimator carries over when the regularization is learned rather than thresholded.
Baselines
The naive reference point is an augmented-state unscented filter whose parameter drift covariance is one constant. That constant is chosen by grid search over the full training stream, using as many filter runs as the grid requires. It answers whether any adaptivity matters.
The strongest existing method on the parameter-tracking question is the excitation-monitored adaptive moving horizon estimator of Nonlinear moving horizon estimation for robust state and parameter estimation. On the modified Chua's circuit with a quiet phase, it reports a parameter RMSE of 0.0209, against 0.1379 for naive moving horizon estimation. We will reimplement it on our own data splits with its published structure. Its freeze thresholds will be tuned by BOHB (BOHB: Robust and Efficient Hyperparameter Optimization at Scale) under a compute budget equal to our method's single pass plus the grid search given to the naive baseline. On the unmodified quiet-phase protocol, beating it means matching or improving on its parameter RMSE. On the shock-augmented variant, beating it means lower parameter RMSE with fewer total filter evaluations. If we cannot reproduce something close to 0.0209 on the original protocol, we will report that before any comparison.
For forecasting on real data, the strongest reported competitors are two methods that use outside inputs without explicit parameters. One is EgPDE-Net on SML2010. It reports RMSE and MAE reductions of 9.85% and 13.98% relative to ETN-ODE, but the file gives no absolute value. The other is ST-ResNet, with an RMSE of 16.89 on TaxiBJ. Both will be trained under the same splits and the same hyperparameter search budget as our method.
Three ablations isolate the key ingredients. Removing $u_t$ from the gate tests whether outside context matters beyond excitation. Setting $L=1$ tests the bidirectional window and reproduces the greedy hypergradient of Online Learning Rate Adaptation with Hypergradient Descent. Replacing the $L$-lag gradient with Koopman extrapolation from Glocal Hypergradient Estimation with Koopman Operator tests whether hindsight from a smoother beats extrapolated hindsight.
Why not just…?
Why not just tune one constant drift rate with Bayesian optimization? This is the most serious threat. If the systems we study have stationary statistics, a well-tuned constant may be enough, and our gate will learn nearly a constant. The evidence against it in general is indirect. Self-Tuning Networks found that its learned schedules outperform their own final values when those values are retrained as constants. The excitation-monitor paper shows that a single fixed regularization fails across quiet and active phases. We will settle the question directly. On the quiet-phase Chua's circuit and on streams with injected shocks, we will compare against the grid- and BOHB-tuned constant at matched compute. If the constant comes within noise of our method, we will report that the gate is unnecessary there.
Why not just use the excitation monitor and skip outside variables? On the original quiet-phase protocol it may well suffice, and we expect parity there. The file's argument is that outside shocks look like excitation. An excitation-only rule cannot tell "the system changed" from "the system was pushed by something we observed." The no-$u_t$ ablation and the shock-augmented benchmarks test exactly this. If the ablation does as well, the outside-context half of the claim fails.
Why not just learn the dynamics end to end with an exogenous neural ODE? EgPDE-Net forecasts well but identifies no physical parameters. Its own authors note a discrete recurrent encoder, and it has no mechanism for deciding when constants change. On forecasting alone it may win. We will report that honestly. Our claim on real data is narrower: competitive forecasts together with interpretable parameter trajectories whose changes line up with logged outside events.
Why not just estimate states first and fit parameters afterwards? In the file, two-stage pipelines degrade sharply with noise. Modeling Latent Non-Linear Dynamical System over Time Series reports decoupled regression baselines collapsing at 25–50% noise on dysts, while joint latent estimation stays stable. We include a two-stage filter-then-regress baseline on dysts mainly to confirm this in our setting.
Proposed Experiments
The first decisive check uses the modified Chua's circuit with its active, quiet, and re-excited phases from Nonlinear moving horizon estimation for robust state and parameter estimation. We extend it with a variant in which observed outside forcing pulses arrive during the quiet phase, and the true parameters change once at a logged regime switch. Metrics are parameter RMSE over time, state tracking error, and the total number of filter evaluations. The method succeeds if it matches the 0.0209 reference on the original protocol, clearly beats both the adaptive moving horizon estimator and the tuned constant on the shock variant, and the no-$u_t$ ablation loses that advantage.
The second check sweeps $L$ on dysts systems with added known forcing at 5–50% observation noise. It measures parameter error and gradient bias against a full-trajectory gradient computed offline. This tests whether a moderate $L$ recovers most of the long-horizon benefit at a small fraction of the memory cost. It also tests whether Koopman extrapolation breaks down on chaotic systems, as its regularity assumption predicts. We also report sensitivity to $\beta$ across two orders of magnitude, given the learning-rate brittleness seen in Bidirectional Learning for Time-series Models with Hidden Units.
The third check uses real data with outside covariates. On SML2010 we use a low-order thermal state-space model driven by solar radiation and heating inputs. On TaxiBJ we use a latent linear state-space model per region with weather and holiday metadata. Metrics are forecast RMSE and MAE against EgPDE-Net and ST-ResNet (16.89 RMSE on TaxiBJ). We also check qualitatively whether the learned drift rate rises around holidays and weather shifts and stays low otherwise. The real-data claim fails if the drift rate is uncorrelated with logged events, or if forecasts fall well behind the black-box models.
References
- Nonlinear moving horizon estimation for robust state and parameter estimation
- Robust online joint state/input/parameter estimation of linear systems
- Dual Estimation and the Unscented Transformation
- Particle Metropolis-Hastings using gradient and Hessian information
- Online Learning Rate Adaptation with Hypergradient Descent
- Glocal Hypergradient Estimation with Koopman Operator
- Bidirectional Learning for Time-series Models with Hidden Units
- Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
- BOHB: Robust and Efficient Hyperparameter Optimization at Scale
- EgPDE-Net: Building Continuous Neural Networks for Time Series Prediction with Exogenous Variables
- Predicting citywide crowd flows using deep spatio-temporal residual networks
- Modeling Latent Non-Linear Dynamical System over Time Series