When deploying an autonomous system into the real world—whether it is a quadrotor navigating wind gusts, an autonomous vehicle tracking a human driver, or a battery management system monitoring cell degradation—the underlying mathematical model of the system is rarely static or perfectly known.
Traditionally, engineers use batch system identification: they collect a large dataset of inputs and outputs, offload it to a powerful computer, and find the parameters that best fit the data using optimization techniques like Ordinary Least Squares (OLS). However, as noted by Emre Özkan et al. in Recursive maximum likelihood identification of jump Markov nonlinear systems, batch processing is fundamentally unsuited for real-time applications due to its massive memory overhead and inability to adapt to sudden environmental shifts.
To build truly adaptive systems, we must shift from batch optimization to recursive parameter estimation. This paradigm allows a system to update its internal model at every single time step using only the latest observation, completely eliminating the need to store historical data.
The Core Mechanism: Compressing History
To understand why recursive estimation works, let us look at the standard Ordinary Least Squares solution for a linear system. To find the parameter vector $\theta$ that maps inputs $X$ to outputs $Y$, we solve:
$$\theta = (X^T X)^{-1} X^T Y$$
If we have been operating for hours, $X$ and $Y$ contain millions of rows. Recomputing this matrix inversion at 100 Hz is computationally impossible on embedded hardware.
Recursive algorithms bypass this bottleneck by maintaining a running summary of the past. Instead of keeping the entire matrix $X$, the algorithm maintains an estimate of the parameter vector $\theta_t$ and a "memory matrix" $P_t$, which represents the inverse covariance of the inputs (essentially, our uncertainty about the parameters).
When a new measurement arrives at time step $t$, the update follows a simple, elegant three-step loop:
- Predict: The system uses its current parameters $\theta_{t-1}$ to predict the next state.
- Calculate Innovation: It measures the actual state and computes the prediction error (the innovation).
- Correct: It updates the parameters by shifting them in the direction that reduces the error, scaled by a gain vector derived from $P_t$.
[ Current Model (θ_t-1) ] ---> Predict Next State
^ |
| v
Update Model <--- Calculate Prediction Error (Innovation) <--- Real-World Observation
This mechanism allows the model to grow smarter with every data point without ever looking backward. For instance, Matthieu Blanke et al. in Online greedy identification of linear dynamical systems demonstrate how an online OLS approach can continuously refine system parameters at every time step, matching the accuracy of complex batch methods with a fraction of the computational footprint.
Managing Uncertainty and Non-Stationarity
In a perfectly stationary world, our uncertainty matrix $P_t$ would eventually shrink to zero, meaning the algorithm has converged and will no longer alter its parameter estimates. But real-world systems are non-stationary: physical properties drift, temperatures fluctuate, and environments change.
If $P_t$ becomes too small, the algorithm becomes "deaf" to new data. To prevent this, researchers use a Forgetting Factor ($\lambda$). The forgetting factor exponentially discounts past observations, ensuring that the estimator always prioritizes recent data.
This approach, known as Forgetting Factor Recursive Least Squares (FFRLS), is highly effective in electrochemical and mechanical monitoring. For example, Xin Lai et al. in Co-Estimation of State-of-Charge and State-of-Health for Lithium-Ion Batteries Considering Temperature and Ageing and Hongyu Pan et al. in Online Broadband Impedance Identification for Lithium-Ion Batteries Based on a Nonlinear Equivalent Circuit Model leverage FFRLS to track the internal impedance of lithium-ion batteries. By discounting old data, their systems can accurately capture rapid changes in battery dynamics caused by temperature spikes or long-term aging without experiencing parameter divergence.
Where RLS tracks parameters by updating an uncertainty covariance matrix, an alternative approach is Online Gradient Descent (OGD). Instead of maintaining a matrix $P_t$, OGD simply takes a step proportional to the gradient of the prediction error. Christos Dimitrakakis in Online statistical estimation for vehicle control applies this to vehicle dynamics, using stochastic gradient descent to continuously learn braking and acceleration limits from noisy operational data.
Expanding the Mathematical Frontier
While classical recursive estimation was designed for simple linear models, modern applications require identifying highly complex, nonlinear, or structured relationships. The field has evolved through several algorithmic innovations:
- Kernel Methods for Nonlinearity: To capture complex nonlinearities without losing the efficiency of recursive linear updates, David Wingate et al. in Kernel Predictive Linear Gaussian models for nonlinear stochastic dynamical systems integrate Kernel Recursive Least-Squares (KRLS). This maps inputs into a high-dimensional feature space where the dynamics appear linear, allowing recursive updates to model nonlinear stochastic systems.
- State-Space Formulations: Instead of treating parameters purely as static weights, they can be modeled as hidden states within a dynamic system. Lucie Daubigney in Single-trial P300 detection with Kalman filtering and SVMs utilizes a recursive state-space filter (a Kalman filter) to adaptively estimate optimal coefficients for real-time brain-computer interfaces.
- Parallelized Multi-Model Estimation: When certain parameters (like time delays) are discrete or non-differentiable, a single filter cannot resolve them. Zhaojian Li 0001 et al. in A New Microscopic Traffic Model Using a Spring-Mass-Damper-Clutch System solve this by running multiple estimation filters in parallel—each assuming a different discrete reaction delay—and dynamically selecting the configuration that minimizes real-time prediction error.
Closing the Loop: Adaptive Control and Safety
Why go to such lengths to estimate parameters online? The ultimate goal is often to feed these parameters directly into a controller, a strategy known as Certainty Equivalence control.
In traditional robust control, engineers must design policies based on the absolute worst-case scenario, leading to highly conservative and inefficient behavior. By estimating parameters online, a system can safely expand its operating envelope based on actual, observed conditions.
[ Physical System ] ---> Real-Time Data ---> [ Online Parameter Estimator ]
^ |
| v
[ Adaptive Controller (MPC) ] <------------------- Updated System Model
This synergy is particularly powerful when paired with Model Predictive Control (MPC) and reinforcement learning:
- Reducing Conservativeness: Claire Tomlin et al. in Reducing Conservativeness in Safety Guarantees by Learning Disturbances Online: Iterated Guaranteed Safe Online Learning demonstrate that by performing regression on real-time operational data to identify environmental disturbances, a robotic controller can safely shrink its paranoid safety margins without risking catastrophic failure.
- Guaranteed Performance Bounds: Alexandre Proutiere et al. in Sub-linear Regret in Adaptive Model Predictive Control combine recursive least squares with robust tube-based MPC to prove that the cost of learning system dynamics online scales sub-linearly ($O(T^{1/2})$), guaranteeing that the adaptive controller eventually performs as well as an "oracle" that knew the system perfectly from the start.
- Handling Unmodeled Dynamics: Hongyu Zhou et al. in Simultaneous System Identification and Model Predictive Control with No Dynamic Regret show how online gradient descent allows autonomous agents to adapt to complex, unmodeled aerodynamic effects on the fly. Similarly, Efe C. Balta et al. in Policy Optimization with Differentiable MPC: Convergence Analysis under Uncertainty prove that updating an MPC framework with the latest recursively learned model drastically improves tuning efficiency and guarantees convergence to an optimal controller even amidst stochastic noise.
By treating system identification not as a preparatory step, but as an active, lifelong component of the control loop, recursive parameter estimation provides the mathematical foundation that allows machines to safely navigate an unpredictable and ever-changing world.
Key Foundational Work
Basic adaptive control theory and its application to robot control-II. (1985) — contributed 2 concept elements to this direction.
Convergence Aspects of Some Robust Estimators Based Upon Prefiltering of the Input-Output Data (1990) — contributed 1 concept elements to this direction.