Online Learning for Supervisory Switching Control

Haoyuan Sun, Ali Jadbabaie

Abstract

We study supervisory switching control for partially-observed linear dynamical systems. The objective is to identify and deploy the best controller for the unknown system by periodically selecting among a collection of $N$ candidate controllers, some of which may destabilize the underlying system. While classical estimator-based supervisory control guarantees asymptotic stability, it lacks quantitative finite-time performance bounds. Conversely, current non-asymptotic methods in both online learning and system identification require restrictive assumptions that are incompatible in a control setting, such as system stability, which preclude testing potentially unstable controllers. To bridge this gap, we propose a novel, non-asymptotic analysis of supervisory control that adapts multi-armed bandit algorithms to a control-theoretic setting. The proposed data-driven algorithm evaluates candidate controllers via scoring criteria that leverage system observability to isolate the effects of state history, enabling both detection of destabilizing controllers and accurate system identification. We present two algorithmic variants with dimension-free, finite-time guarantees, where each identifies the most suitable controller in $\mathcal{O}(N \log N)$ steps, while simultaneously achieving finite $L_2$-gain with respect to system disturbances.

Introduction

In complex engineering domains such as autonomous vehicle fleets, smart power grids, and epidemic management, a single static controller is often insufficient to handle varying operating conditions or unknown system parameters. Engineers frequently turn to supervisory switching control, a high-level strategy that selects the most appropriate controller from a library of $N$ candidates based on real-time feedback.

Despite its practical utility, two major theoretical hurdles have persisted. First, the "Stability Catch-22": most modern non-asymptotic learning methods assume the system is already stable to provide fast guarantees. However, in a switching control context, the supervisor must explore unknown controllers, some of which may be destabilizing. Second, classical methods focus on asymptotic stability—guaranteeing the system will eventually settle—but fail to provide finite-time bounds on performance or deviation during the learning phase. This paper bridges these gaps by providing a non-asymptotic analysis that ensures stability even while exploring potentially dangerous controllers.

The Supervisory Control Framework

The core of the proposed approach is a data-driven supervisor that manages a collection of candidate controllers. The system is modeled as a partially-observed linear dynamical system where the plant $(C_\star, A_\star, B_\star)$ is unknown, but is assumed to match one of $N$ candidate models.

Closed-loop system architecture for supervisory switching control.

As shown in the figure above, the supervisor selects a controller $K$ at each epoch $t$. The challenge lies in the fact that the state at the end of one control episode becomes the initial state for the next, creating a temporal dependency that violates the independent-trial assumptions of standard machine learning algorithms like Multi-Armed Bandits (MAB).

Methodology: Observability-Based Scoring

To apply MAB techniques to this state-dependent problem, the authors introduce an observability-based scoring mechanism. This "secret sauce" allows the algorithm to decouple the current controller's performance from the "hangover" effects of previous controllers.

  1. State Reconstruction: The algorithm uses the initial steps of an episode to estimate the initial state ($\hat{x}_1$) using the system's observability properties.
  2. Transient Removal: By subtracting the predicted transient response (the influence of $\hat{x}_1$) from the observed output, the supervisor can isolate the controller's intrinsic performance.

Evaluation Criteria

The algorithm evaluates the active controller using two primary criteria:

  • Instability Detection: This criterion monitors the residual error between predicted and observed outputs. If the error grows beyond a specific threshold, the controller is flagged as destabilizing and rejected.
  • System Identification: Using Ordinary Least Squares (OLS), the algorithm compares the system's Markov parameters against the candidate model to find the best match.

By treating the selection process as an Upper Confidence Bound (UCB) problem, the supervisor balances the need to explore different controllers with the need to exploit the one that appears most stable and accurate.

Key Results and Stability Guarantees

The proposed framework yields several significant theoretical breakthroughs:

  • Sample Complexity: The algorithm identifies the optimal controller in $\mathcal{O}(N \log N)$ episodes. This represents a substantial improvement over prior non-asymptotic works that often required exponential time relative to the number of candidates.
  • Finite $L_2$-gain Stability: The authors provide a rigorous proof that the system remains stable in an $L_2$ sense throughout the learning process. Even if the supervisor tests a destabilizing controller, the algorithm detects the divergence quickly enough to switch away before the system state exceeds a safe energy bound.
  • Dimension-Free Bounds: Remarkably, the performance guarantees do not depend on the dimension of the state space, suggesting that the method can scale to high-dimensional industrial systems.

Conclusion and Future Work

This research provides a robust mathematical foundation for combining online learning with switching control. By exploiting the predictable way in which unstable systems diverge, the algorithm turns a potential hazard into a useful signal for rapid controller rejection.

Future work aims to address current limitations, such as the restriction to linear systems and the "active evaluation" constraint. The authors suggest that cross-evaluation—using data from one controller to evaluate all $N$ candidates simultaneously—could further accelerate the identification process. Additionally, extending these finite-time guarantees to nonlinear systems remains a critical frontier for the field.

Summary updated

Abstract

We study supervisory switching control for partially-observed linear dynamical systems. The objective is to identify and deploy the best controller for the unknown system by periodically selecting among a collection of $N$ candidate controllers, some of which may destabilize the underlying system. While classical estimator-based supervisory control guarantees asymptotic stability, it lacks quantitative finite-time performance bounds. Conversely, current non-asymptotic methods in both online learning and system identification require restrictive assumptions that are incompatible in a control setting, such as system stability, which preclude testing potentially unstable controllers. To bridge this gap, we propose a novel, non-asymptotic analysis of supervisory control that adapts multi-armed bandit algorithms to a control-theoretic setting. The proposed data-driven algorithm evaluates candidate controllers via scoring criteria that leverage system observability to isolate the effects of state history, enabling both detection of destabilizing controllers and accurate system identification. We present two algorithmic variants with dimension-free, finite-time guarantees, where each identifies the most suitable controller in $\mathcal{O}(N \log N)$ steps, while simultaneously achieving finite $L_2$-gain with respect to system disturbances.
BibTeX
@misc{sun2026online,
  author = {Haoyuan Sun and Ali Jadbabaie},
  title = {{Online Learning for Supervisory Switching Control}},
  year = {2026},
  eprint = {2603.14762},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.14762},
}
Figure