Towards targeted exploration for non-stochastic disturbances

Janani Venkatasubramanian, Johannes Köhler, Mark Cannon, Frank Allgöwer

TL;DR

This paper introduces a novel targeted exploration strategy for linear systems facing deterministic, energy-bounded disturbances rather than standard stochastic noise. By deriving a data-dependent sufficient condition and formulating it as a semidefinite program (SDP), the authors optimize multisine inputs to guarantee accurate parameter estimation with minimal energy, even in the presence of unmodeled nonlinearities.

What Problem Does This Paper Address?

To control a system effectively, you need an accurate mathematical model. "Targeted exploration" (or optimal experiment design) is the process of strategically injecting inputs into a system to gather the most informative data possible, minimizing model uncertainty.

Traditionally, researchers assume that system noise is stochastic—meaning it is independent, zero-mean, and follows a known distribution (like Gaussian noise). However, real-world systems often suffer from unmodeled dynamics, calibration errors, or nonlinearities. These physical mismatches act as deterministic, non-stochastic disturbances. Applying standard stochastic exploration strategies in these scenarios can lead to overconfident and highly inaccurate parameter estimates. This paper addresses the gap by designing an optimal exploration strategy tailored specifically for systems with non-stochastic, energy-bounded disturbances.

Methods

The authors develop a robust framework by combining classical set-membership identification with modern convex optimization:

  1. Non-Stochastic Uncertainty Bound: The method builds on Fogel’s (1979) data-dependent uncertainty bound, which defines an exact ellipsoidal region of "non-falsified" parameters based on an energy bound ($\gamma_w$) on the disturbances.
  2. Sufficient Condition for Accuracy: The authors derive a mathematical condition (Theorem 4) ensuring that the resulting parameter estimate falls within a user-defined closeness matrix ($D_{des}$).
  3. Convex Relaxation: Because the exact condition is non-convex and computationally intractable, they apply a clever linearization technique (using Schur complements) to transform the constraints.
  4. Semidefinite Program (SDP) for Multisine Inputs: The exploration inputs are structured as multisine signals (a sum of sinusoids at distinct frequencies). The linearized conditions are embedded into an SDP that minimizes the total input energy while guaranteeing the tracking performance.
  5. Iterative Refinement: They introduce an iterative algorithm (Algorithm 1) that sequentially updates the relaxation matrices to eliminate conservatism and converge on an optimal input profile.

Datasets

Not discussed in the paper. The authors evaluate their method using a synthetic numerical simulation of a 4-state linear time-invariant (LTI) system rather than an empirical dataset.

Key Insights and Ideas

  • Data-Dependent Scaling: In stochastic experiment design, the size of the confidence ellipsoid is typically fixed beforehand by probability thresholds. In the non-stochastic setting, the scaling factor ($G$) of the ellipsoid is explicitly data-dependent—it changes based on the actual states and inputs observed during the experiment.
  • Time Doesn't Heal All Bounds: Under stochastic noise, running an experiment longer with minimal excitation will eventually drive parameter uncertainty to zero. For energy-bounded deterministic noise, simply increasing time is not enough; you must actively design the input energy distribution to fight the worst-case disturbance.
  • Different Noise, Different Strategy: The optimal way to distribute energy across frequencies changes drastically when moving from a stochastic assumption to a robust, energy-bounded assumption.

Key Results

The authors compared their non-stochastic targeted exploration strategy against a conventional stochastic strategy using a 4-state system corrupted by an unmodeled nonlinearity ($w_k = c[-\cos(x_{k,1})\, 0\, 0\, 0]^\top$).

  • Energy Distribution: The two strategies yielded entirely different input profiles. As shown in Figure 1, the non-stochastic approach concentrates its relative energy at lower frequencies ($\omega_i = 0.1$ and $0.2$), whereas the stochastic approach peaks at a higher frequency ($\omega_i = 0.3$).
Optimal relative energy over different frequencies for non-stochastic and stochastic targeted exploration.
  • Estimation Accuracy: When tested against the deterministic nonlinearity, the non-stochastic exploration strategy achieved a significantly lower parameter estimation error than its stochastic counterpart:
  • Non-stochastic Strategy Error: $0.0257$
  • Stochastic Strategy Error: $0.0389$

This confirms that accounting for the energy-bounded nature of unmodeled dynamics prevents the system from being misled by deterministic errors.

Limitations and Future Work

  • Approximations for Tractability: To solve the SDP, the authors rely on standard simplifications: they assume future states evolve according to a rough initial parameter estimate ($\hat{\theta}_0$) and neglect the direct impact of the noise on future states during the planning phase.
  • Open-Loop Stability: The current open-loop input design assumes the underlying system is stable (Schur stable). For unstable systems, a robustly stabilizing feedback controller must be integrated into the loop.
  • Future Directions: The authors note that future work will focus on eliminating these approximations to robustly account for noise and parametric uncertainty without simplifications. They also plan to adapt this targeted exploration design to directly guarantee robust control performance.

Summary updated

Abstract

We present a novel targeted exploration strategy for linear time-invariant systems without stochastic assumptions on the noise, i.e., without requiring independence or zero mean, allowing for deterministic model misspecifications. This work utilizes classical data-dependent uncertainty bounds on the least-squares parameter estimates in the presence of energy-bounded noise. We provide a sufficient condition on the exploration data that ensures a desired error bound on the estimated parameter. Using common approximations, we derive a semidefinite program to compute the optimal sinusoidal input excitation. Finally, we highlight the differences and commonalities between the developed non-stochastic targeted exploration strategy and conventional exploration strategies based on classical identification bounds through a numerical example.
BibTeX
@inproceedings{venkatasubramanian2024towards,
  author = {Janani Venkatasubramanian and Johannes Köhler and Mark Cannon and Frank Allgöwer},
  title = {{Towards targeted exploration for non-stochastic disturbances}},
  booktitle = {IFAC-PapersOnLine},
  year = {2024},
  doi = {10.1016/j.ifacol.2024.08.588},
}
Figure