Data-Driven Predictive Control Using Closed-Loop Data: An Instrumental Variable Approach

Yibo Wang, Yiwen Qiu, Malika Sader, Dexian Huang, Chao ShangIEEE Control. Syst. Lett 2023 · OpenReview

TL;DR

This paper introduces a novel data-driven predictive control (DDPC) method that can learn from system data collected while the system is already under feedback control. By employing instrumental variables (IVs) inspired by closed-loop subspace identification, the method overcomes the limitations of traditional DDPC, which requires open-loop data. This advancement allows for more practical and safer implementation of advanced control strategies in real-world systems.

What Problem Does This Paper Address?

Traditional data-driven predictive control (DDPC) methods are powerful because they can learn control policies directly from data, bypassing the need for explicit system modeling. However, a significant hurdle is that these methods typically require data collected during "open-loop" operation, where the system's inputs are deliberately varied to excite its dynamics. This is often impractical or unsafe in real-world industrial settings where systems must remain under some form of feedback control for stability and operational reasons. When data is collected under closed-loop control, the feedback mechanism introduces correlations between inputs and noise, which biases the predictions and renders standard DDPC ineffective. This paper tackles the challenge of making DDPC work effectively using only closed-loop data.

Methods

The core innovation of this paper lies in the use of Instrumental Variables (IVs) to address the bias introduced by closed-loop data. The authors draw inspiration from closed-loop subspace identification techniques. They propose two specific forms of IVs:

  1. Future Feedback Reference: This IV leverages the system's reference signal, which is often available and can help disentangle the dynamics of the controller from the plant itself.
  2. Controller's Left Coprime Factorization (LCF): A more sophisticated IV is derived from the controller's internal structure using its left coprime factorization. This method aims to more effectively cancel out noise and retain crucial information from the input/output data.

Furthermore, the paper introduces a novel IV-inspired regularizer for the DDPC formulation. This regularizer helps to strike a balance between minimizing the control cost (i.e., making the system perform well) and fitting the collected data using a weighted least-squares approach. This regularization is crucial for improving tracking performance. The proposed method is presented as a convex relaxation of existing control problems, making it computationally tractable.

Datasets

The paper evaluates its proposed method using two main types of data:

  • Simulated Open-Loop Process Data: A standard system with known matrices (A, B, C, D, K) is used to generate data for initial testing and comparison with other methods.
  • Simulated Industrial Furnace Data: A high-fidelity simulation of a two-input, two-output industrial tubular furnace is used for a more realistic application scenario. Data is collected under closed-loop control with a pre-tuned stabilizing controller.

Key Insights and Ideas

  • Closed-loop data is not inherently "bad" for control, but it requires smarter techniques: The paper demonstrates that with the right tools (IVs), closed-loop data can be effectively utilized for advanced control design, opening up new possibilities for real-world applications.
  • The "future reference" is a powerful, yet underutilized, signal for IVs in DDPC: Using the reference signal as an IV helps to disentangle controller and plant dynamics, a critical step for successful closed-loop DDPC.
  • A combination of IVs is better: The paper shows that using both the future reference and an IV derived from the controller's structure leads to superior performance compared to using either one alone.
  • Regularization is key for balancing performance and data fitting: The proposed IV-inspired regularizer allows for a trade-off between achieving optimal control objectives and accurately reflecting the observed data, leading to improved overall performance.

Key Results

The proposed DDPC method using instrumental variables (DDPC-IV) and its regularized version (RDDPC-IV) consistently outperformed existing methods like standard Subspace Predictive Control (SPC) and simpler DDPC variants (DDPC-IV1, DDPC-IV2) in simulations.

  • Improved Performance: Across various noise levels, DDPC-IV and RDDPC-IV showed significantly better control performance, as measured by a cost index that penalizes deviations from the reference and control effort.
  • Effectiveness of Regularization: The RDDPC-IV, with its IV-inspired regularizer, generally performed better than the non-regularized DDPC-IV, especially when the regularization parameter was tuned appropriately.
  • Industrial Furnace Application: In the simulated tubular furnace example, the proposed DDPC-IV method achieved better tracking of the outlet temperature and maintained the oxygen content near its desired value, outperforming both a generic SPC and the initial stabilizing controller.

Limitations and Future Work

  • Controller Knowledge Assumption: The proposed method assumes that the controller's dynamics (Ac, Bc, Cc, Dc) are known. This might not always be the case in practice.
  • Tuning of Regularization Parameter: The performance of the regularized DDPC-IV depends on the choice of the regularization parameter (λ). While the paper explores a range of values, optimal tuning for specific applications might require further investigation.
  • Future Work: The authors suggest exploring more sophisticated IV choices and regularization strategies. Additionally, extending the method to handle nonlinear systems and more complex industrial scenarios would be valuable future research directions.

Summary updated

Abstract

Current data-driven predictive control (DDPC) methods heavily rely on data collected in open-loop operation with elaborate design of inputs. However, due to safety or economic concerns, systems may have to be under feedback control, where only closed-loop data are available. In this context, it remains challenging to implement DDPC using closed-loop data. In this letter, we propose a new DDPC method using closed-loop data by means of instrumental variables (IVs). We point out that the original DDPC fails to represent all admissible trajectories due to feedback control. By drawing from closed-loop subspace identification, the use of two forms of IVs is suggested to address this issue and the correlation between inputs and noise. Furthermore, a new DDPC formulation with a novel IV-inspired regularizer is proposed, where a balance between control cost minimization and weighted least-squares data fitting can be made for improvement of control performance. Numerical examples and application to a simulated industrial furnace showcase the improved performance of the proposed DDPC based on closed-loop data.
BibTeX
@inproceedings{wang2023data,
  author = {Yibo Wang and Yiwen Qiu and Malika Sader and Dexian Huang and Chao Shang},
  title = {{Data-Driven Predictive Control Using Closed-Loop Data: An Instrumental Variable Approach}},
  booktitle = {IEEE Control. Syst. Lett},
  year = {2023},
  url = {https://openreview.net/forum?id=5ASDLGyqzF},
}
Figure