Abstract
We propose an algorithm for designing optimal inputs for on-line Bayesian identification of stochastic non-linear state-space models. The proposed method relies on minimization of the posterior Cramér Rao lower bound derived for the model parameters, with respect to the input sequence. To render the optimization problem computationally tractable, the inputs are parametrized as a multi-dimensional Markov chain in the input space. The proposed approach is illustrated through a simulation example.
Introduction
Identifying the internal parameters of a system—such as the drag coefficient of a drone or the reaction rate in a chemical plant—often requires observing how the system responds to specific inputs. In the context of non-linear state-space models (SSMs), this task is notoriously difficult due to hidden states (noisy outputs) and the computational complexity of Bayesian identification, which involves high-dimensional integrals without analytical solutions.
Existing input design methods often struggle with the "curse of dimensionality" inherent in real-world stochastic processes. This paper introduces a novel algorithm to find the optimal input sequence to minimize parameter estimation error by "steering" experiments toward maximum information gain.
Methodology
The proposed strategy consists of three core components designed to ensure computational tractability:
- Objective Function (PCRLB): Instead of minimizing the Mean Square Error (MSE) directly, which is computationally formidable, the method minimizes the Posterior Cramér-Rao Lower Bound (PCRLB). This theoretical floor represents the minimum possible variance any unbiased estimator can achieve.
- Monte Carlo (MC) Approximation: To handle the complex integrals over states and parameters within the PCRLB, the authors employ MC sampling. This transforms the calculus problem into a manageable summation of random samples.
- Markov Chain Input Parametrization: To avoid optimizing every input value at every time step, the input sequence is modeled as a multi-dimensional Markov chain. By optimizing transition probabilities between discrete input levels rather than specific values, the number of optimization variables is drastically reduced while naturally accommodating physical constraints.
Experimental Results
The algorithm was evaluated using a simulated univariate, non-stationary stochastic SSM, a standard benchmark in non-linear filtering. The goal was to estimate a vector of four unknown parameters ($\theta = [a, b, c, d]$) over 100 time steps.
As shown in the figures above, the optimized input (Case 3) consistently achieved lower error bounds and lower actual MSE compared to other designs. Specifically, the trace of the lower bound for Case 3 remains below the other cases throughout the experiment duration, as highlighted in the magnified inset.
Quantitative results in the table above confirm that Case 3 achieved the lowest objective function value (0.36) and the lowest cumulative trace of the MSE (1.25) when tested with a Sequential Monte Carlo (SMC) estimator. In contrast, the standard pseudo-random binary signal (Case 4) resulted in significantly higher values (0.51 and 2.02, respectively).
Key Insights and Future Work
A significant advantage of this approach is its estimator independence; the designed input optimizes the information potential of the experiment itself, regardless of whether a Particle Filter or MCMC is used for final identification. By discretizing the input space, the authors transform a high-dimensional trajectory optimization problem into a feasible stochastic programming problem.
Future work will address the challenge of local minima in the non-smooth cost function. The authors suggest that moving toward stochastic gradient-based methods may improve convergence speed and efficiency for even more complex, higher-dimensional systems.
Summary updated