Online System Identification via Recursive Parameter Estimation

55 concepts52 papers1 proposal

When deploying an autonomous system into the real world—whether it is a quadrotor navigating wind gusts, an autonomous vehicle tracking a human driver, or a battery management system monitoring cell degradation—the underlying mathematical model of the system is rarely static or perfectly known.

Traditionally, engineers use batch system identification: they collect a large dataset of inputs and outputs, offload it to a powerful computer, and find the parameters that best fit the data using optimization techniques like Ordinary Least Squares (OLS). However, as noted by Emre Özkan et al. in Recursive maximum likelihood identification of jump Markov nonlinear systems, batch processing is fundamentally unsuited for real-time applications due to its massive memory overhead and inability to adapt to sudden environmental shifts.

To build truly adaptive systems, we must shift from batch optimization to recursive parameter estimation. This paradigm allows a system to update its internal model at every single time step using only the latest observation, completely eliminating the need to store historical data.


The Core Mechanism: Compressing History

To understand why recursive estimation works, let us look at the standard Ordinary Least Squares solution for a linear system. To find the parameter vector $\theta$ that maps inputs $X$ to outputs $Y$, we solve:

$$\theta = (X^T X)^{-1} X^T Y$$

If we have been operating for hours, $X$ and $Y$ contain millions of rows. Recomputing this matrix inversion at 100 Hz is computationally impossible on embedded hardware.

Recursive algorithms bypass this bottleneck by maintaining a running summary of the past. Instead of keeping the entire matrix $X$, the algorithm maintains an estimate of the parameter vector $\theta_t$ and a "memory matrix" $P_t$, which represents the inverse covariance of the inputs (essentially, our uncertainty about the parameters).

When a new measurement arrives at time step $t$, the update follows a simple, elegant three-step loop:

  1. Predict: The system uses its current parameters $\theta_{t-1}$ to predict the next state.
  2. Calculate Innovation: It measures the actual state and computes the prediction error (the innovation).
  3. Correct: It updates the parameters by shifting them in the direction that reduces the error, scaled by a gain vector derived from $P_t$.
  [ Current Model (θ_t-1) ] ---> Predict Next State
             ^                            |
             |                            v
       Update Model <--- Calculate Prediction Error (Innovation) <--- Real-World Observation

This mechanism allows the model to grow smarter with every data point without ever looking backward. For instance, Matthieu Blanke et al. in Online greedy identification of linear dynamical systems demonstrate how an online OLS approach can continuously refine system parameters at every time step, matching the accuracy of complex batch methods with a fraction of the computational footprint.


Managing Uncertainty and Non-Stationarity

In a perfectly stationary world, our uncertainty matrix $P_t$ would eventually shrink to zero, meaning the algorithm has converged and will no longer alter its parameter estimates. But real-world systems are non-stationary: physical properties drift, temperatures fluctuate, and environments change.

If $P_t$ becomes too small, the algorithm becomes "deaf" to new data. To prevent this, researchers use a Forgetting Factor ($\lambda$). The forgetting factor exponentially discounts past observations, ensuring that the estimator always prioritizes recent data.

This approach, known as Forgetting Factor Recursive Least Squares (FFRLS), is highly effective in electrochemical and mechanical monitoring. For example, Xin Lai et al. in Co-Estimation of State-of-Charge and State-of-Health for Lithium-Ion Batteries Considering Temperature and Ageing and Hongyu Pan et al. in Online Broadband Impedance Identification for Lithium-Ion Batteries Based on a Nonlinear Equivalent Circuit Model leverage FFRLS to track the internal impedance of lithium-ion batteries. By discounting old data, their systems can accurately capture rapid changes in battery dynamics caused by temperature spikes or long-term aging without experiencing parameter divergence.

Where RLS tracks parameters by updating an uncertainty covariance matrix, an alternative approach is Online Gradient Descent (OGD). Instead of maintaining a matrix $P_t$, OGD simply takes a step proportional to the gradient of the prediction error. Christos Dimitrakakis in Online statistical estimation for vehicle control applies this to vehicle dynamics, using stochastic gradient descent to continuously learn braking and acceleration limits from noisy operational data.


Expanding the Mathematical Frontier

While classical recursive estimation was designed for simple linear models, modern applications require identifying highly complex, nonlinear, or structured relationships. The field has evolved through several algorithmic innovations:

  • Kernel Methods for Nonlinearity: To capture complex nonlinearities without losing the efficiency of recursive linear updates, David Wingate et al. in Kernel Predictive Linear Gaussian models for nonlinear stochastic dynamical systems integrate Kernel Recursive Least-Squares (KRLS). This maps inputs into a high-dimensional feature space where the dynamics appear linear, allowing recursive updates to model nonlinear stochastic systems.
  • State-Space Formulations: Instead of treating parameters purely as static weights, they can be modeled as hidden states within a dynamic system. Lucie Daubigney in Single-trial P300 detection with Kalman filtering and SVMs utilizes a recursive state-space filter (a Kalman filter) to adaptively estimate optimal coefficients for real-time brain-computer interfaces.
  • Parallelized Multi-Model Estimation: When certain parameters (like time delays) are discrete or non-differentiable, a single filter cannot resolve them. Zhaojian Li 0001 et al. in A New Microscopic Traffic Model Using a Spring-Mass-Damper-Clutch System solve this by running multiple estimation filters in parallel—each assuming a different discrete reaction delay—and dynamically selecting the configuration that minimizes real-time prediction error.

Closing the Loop: Adaptive Control and Safety

Why go to such lengths to estimate parameters online? The ultimate goal is often to feed these parameters directly into a controller, a strategy known as Certainty Equivalence control.

In traditional robust control, engineers must design policies based on the absolute worst-case scenario, leading to highly conservative and inefficient behavior. By estimating parameters online, a system can safely expand its operating envelope based on actual, observed conditions.

[ Physical System ] ---> Real-Time Data ---> [ Online Parameter Estimator ]
       ^                                                    |
       |                                                    v
[ Adaptive Controller (MPC) ] <------------------- Updated System Model

This synergy is particularly powerful when paired with Model Predictive Control (MPC) and reinforcement learning:

By treating system identification not as a preparatory step, but as an active, lifelong component of the control loop, recursive parameter estimation provides the mathematical foundation that allows machines to safely navigate an unpredictable and ever-changing world.

Key Foundational Work

Basic adaptive control theory and its application to robot control-II. (1985) — contributed 2 concept elements to this direction.
Convergence Aspects of Some Robust Estimators Based Upon Prefiltering of the Input-Output Data (1990) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_online_parameter_estimation_recursive_pa_3f6b4630,
  title = {online parameter estimation (recursive parameter estimation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/online-parameter-estimation-recursive-parameter-estimation/rdr_3f6b4630818040169f3fc1a6a1d5ee00},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_online_parameter_estimation_recursive_pa_3f6b4630_v1,
  title = {online parameter estimation (recursive parameter estimation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/online-parameter-estimation-recursive-parameter-estimation/rdr_3f6b4630818040169f3fc1a6a1d5ee00/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake