Controller Design via Experimental Exploration With Robustness Guarantees

Tobias Holicki, Carsten W. Scherer, Sebastian TrimpeIEEE Control. Syst. Lett 2021 · OpenReview

TL;DR

This paper introduces a novel approach to designing controllers for partially unknown linear systems by combining experimental exploration with robust control guarantees. It proposes a systematic way to parameterize candidate controllers that are guaranteed to be safe (i.e., stabilize the system) and uses this to efficiently tune controllers based on real-world experiments, leading to improved performance.

What Problem Does This Paper Address?

Designing controllers for systems where we don't have a perfect model is a significant challenge in control engineering. Traditional methods often require a precise model, while purely data-driven or learning-based approaches can struggle with safety guarantees. Specifically, when tuning controllers through experimentation, there's a risk of implementing a controller that destabilizes the system, leading to catastrophic outcomes. Existing methods for learning controllers often lack explicit safety guarantees or require extensive prior knowledge. This paper aims to bridge this gap by providing a framework that ensures safety throughout the learning process while still leveraging experimental data to improve performance and reduce uncertainty about the system.

Methods

The core of the paper lies in a systematic controller parametrization technique. They leverage the Linear Fractional Representation (LFR) framework from robust control to model the system. This allows them to explicitly separate known parts of the system from unknown or uncertain components. Prior knowledge about the system is encoded in how this LFR is structured.

The key innovation is a controller parametrization that generates a set of controllers guaranteed to be robustly stabilizing. This is achieved by:

  1. Partitioning the uncertainty set: The unknown part of the system (represented by a set of matrices Δ) is divided into smaller subsets.
  2. Robust multi-objective synthesis: For each subset, they design a controller that is robustly stabilizing for that subset and optimizes a worst-case performance metric. This is done using Linear Matrix Inequalities (LMIs) and multiplier relaxations, which provide upper bounds on the robust performance.
  3. Parametrization: This process yields a mapping from indices (representing the partitions) to safe controllers. The experimental cost function is then evaluated for these parameterized controllers.

This approach ensures that any controller generated by the parametrization is safe to implement. The algorithm then iteratively refines the partitions and controller candidates to improve performance.

Datasets

The paper uses modified examples from the COMPleib library [27] for numerical illustrations. These are not "datasets" in the machine learning sense but rather benchmark systems designed to test control algorithms. The authors note that COMPleib doesn't specifically include robust control examples, so they adapted existing ones.

Key Insights and Ideas

  • Safety First: The paramount importance of guaranteeing system stability during experimental controller tuning is highlighted. The proposed parametrization inherently ensures safety, unlike some other learning-based methods.
  • LFR for Prior Knowledge: The LFR framework is a powerful tool for incorporating existing knowledge about a system's structure into the controller design process, which is often overlooked in learning-based control.
  • Parametrization as a Bridge: The controller parametrization acts as a crucial bridge between robust control theory (for safety) and experimental exploration (for performance improvement).
  • Reducing the "Price of Safety": By systematically refining the uncertainty set and controller candidates, the paper shows how to reduce the performance gap that arises from enforcing safety guarantees.
  • Coordinate-Like Descent: The proposed algorithm uses a coordinate-like descent strategy for refining the controller parametrization, making it applicable to higher-dimensional problems.

Key Results

The paper demonstrates the effectiveness of their approach through numerical examples. They show that their algorithm, when applied to modified benchmark systems, achieves performance levels comparable to or better than standard robust design methods, and often significantly better than a baseline learning approach that lacks explicit safety guarantees.

For instance, in Table I, Algorithm 1 (using their parametrization) achieves performance levels (γ) that are often close to the nominal optimal performance (Ynom) and significantly better than the robust performance upper bound (Vrp), while ensuring safety. This is achieved with a limited number of cost evaluations (e.g., 36 iterations).

Limitations and Future Work

  • Convexity of Uncertainty Set: The paper acknowledges that the uncertainty set A is not guaranteed to be convex. They suggest extending the approach by expressing A as a union of convex sets and using robust stabilization objectives for each, though this increases computational burden.
  • Extension to Output Feedback: The authors mention that the concept can be extended to output feedback control using existing robust control techniques.
  • H2-Norm and Discrete-Time: They also note that the framework can be adapted for H2-norm cost criteria and for discrete-time systems.
  • Dynamic/Nonlinear Uncertainties: While the LFR framework can handle more complex uncertainties (time-varying, dynamic, nonlinear), the current work focuses on parametric uncertainties. Future work could explore these more complex scenarios.

Summary updated

Abstract

For a partially unknown linear systems, we present a systematic control design approach based on generated data from measurements of closed-loop experiments with suitable test controllers. These experiments are used to improve the achieved performance and to reduce the uncertainty about the unknown parts of the system. This is achieved through a parametrization of auspicious controllers with convex relaxation techniques from robust control, which guarantees that their implementation on the unknown plant is safe. This approach permits to systematically incorporate available prior knowledge about the system by employing the framework of linear fractional representations.
BibTeX
@inproceedings{holicki2021controller,
  author = {Tobias Holicki and Carsten W. Scherer and Sebastian Trimpe},
  title = {{Controller Design via Experimental Exploration With Robustness Guarantees}},
  booktitle = {IEEE Control. Syst. Lett},
  year = {2021},
  url = {https://openreview.net/forum?id=5VGmH_kMHRY},
}
Figure