Sample Efficient Exploration for Nonlinear System Identification

22 concepts8 papers1 proposal

In linear system identification, injecting independent random noise into a system's inputs is a highly effective tactic. Because linear dynamics scale uniformly across the entire state space, any direction of random exploration yields globally useful data.

Nonlinear systems break this convenient property. In a nonlinear landscape, random noise can leave a system trapped in uninformative local basins, never exposing the underlying physics governing other regions. To learn these systems efficiently without wasting expensive physical samples, recent research focuses on a core challenge: how can an agent intelligently navigate a nonlinear state space to collect the most informative data, and when can we bypass the computational overhead of complex planning?

The Geometry of Information in Nonlinear Systems

When dealing with unknown nonlinear dynamics, standard exploration techniques like adding white noise or binary chirps often fail. As noted by Solowjow et al., passive noise cannot target the specific, high-uncertainty regions of a state space where a model's predictions are weakest.

To resolve parameter uncertainty, an algorithm must actively force the system into regimes that expose the underlying dynamics. Kakade et al. demonstrate that navigating this uncertainty requires a strategic approach to continuous control, modeling unknown dynamics within reproducing kernel Hilbert spaces to systematically balance exploration and exploitation.

This strategic approach is known as active exploration. Instead of treating data collection as a passive byproduct of control, active learning frameworks treat exploration as an explicit optimization problem. By strategically planning and executing trajectories, an agent can leverage the system's inherent controllability to steer into diverse, informative regions of the feature space. Mania et al. show that this deliberate trajectory planning allows nonlinear system identification to achieve finite-time guarantees and statistical estimation rates comparable to standard linear regression.

Passive Exploration (Random Noise)
State Space: [ Low Uncertainty -> Trapped in Local Basin -> High Uncertainty (Unvisited) ]

Active Exploration (Targeted Trajectories)
State Space: [ Low Uncertainty --------Planned Trajectory-------> High Uncertainty (Resolved) ]

Crucially, this purposeful navigation is vital for practical tasks like model-based reinforcement learning. As Wagenmaker et al. point out, an agent rarely needs a globally perfect model of the world; it only needs to understand the dynamics that govern task-relevant behaviors. Active exploration allows the controller to ignore irrelevant parameters and focus its sample budget entirely on the state-space regimes critical to the objective.

The Local Linearization Shortcut

While active trajectory planning provides strong theoretical guarantees, calculating the globally optimal path through a complex, nonlinear space is computationally expensive. To make these methods practical for real-time embedded systems, researchers utilize a powerful mathematical shortcut: local linearization.

The FLEX algorithm, introduced by Blanke et al., demonstrates that an agent can drastically reduce computational overhead by linearizing both the system dynamics and the model parameters at the current operating point. This allows the system to use classical, lightweight optimal experimental design techniques to greedily select the next most informative action.

Why does this local approximation work so well without sacrificing the accuracy of the global nonlinear model? The intuition is uncovered by Stamouli et al. through a non-asymptotic analysis of the Quadratic Prediction Error Method (QPEM).

By applying a Taylor expansion to the system's prediction errors, the authors decompose the error into two parts:

  1. A dominant, linearized error term.
  2. Higher-order nonlinear residuals.

They prove that once an initial data-gathering phase pushes the parameter estimator into a local neighborhood around the true value, the optimization landscape begins to behave as if it were linear. Crucially, the higher-order nonlinear residuals decay significantly faster than the primary linearized error term. This rapid decay means that the complex nonlinearities effectively vanish from the estimation perspective, allowing localized, computationally efficient strategies to recover the optimal convergence rates of linear systems.

When Random Noise Suffices

Given the clear advantages of active planning and local linearization, it is natural to wonder whether active exploration is an absolute necessity for nonlinear systems. Musavi et al. challenge the assumption that nonlinear systems always require complex, strategic input selection.

The authors show that if a system's feature functions are real-analytic—a broad class of smooth functions that includes polynomials, exponentials, and trigonometric waves—simple, non-active random noise injection is entirely sufficient to achieve an optimal parameter convergence rate of $O(1/\sqrt{T})$.

The intuition behind this result lies in the rigid mathematical structure of analytic functions. A real-analytic function is completely determined by its behavior in any local neighborhood. Because of this property, the data collected by passive, random perturbations cannot easily be confined to an uninformative subspace. The random noise excites the system enough to propagate information across the entire analytic landscape, bypassing the need for deliberate, task-specific trajectory planning and eliminating the computational overhead of active control.

Key Foundational Work

Active Learning for Nonlinear System Identification with Guarantees (2022) — contributed 8 concept elements to this direction.
Actively Learning Gaussian Process Dynamics (2020) — contributed 2 concept elements to this direction.
Information Theoretic Regret Bounds for Online Nonlinear Control (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_active_exploration_planning_6aca58fa,
  title = {active exploration planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/active-exploration-planning/rdr_6aca58fa48ce4037ac8add9c70b44a00},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_active_exploration_planning_6aca58fa_v1,
  title = {active exploration planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/active-exploration-planning/rdr_6aca58fa48ce4037ac8add9c70b44a00/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake