TL;DR
This paper tackles the challenge of identifying dynamic systems operating in closed-loop, where standard identification methods struggle due to insufficient excitation. The authors propose novel methods that generate "dual control" signals, balancing the need for system excitation with minimal disruption to normal operation, specifically tailored to the intended application of the identified model.
What Problem Does This Paper Address?
In industrial settings, many control systems rely on accurate models of the plant (the system being controlled). Model Predictive Control (MPC) is a prime example, where controllers use a model to predict future behavior and optimize control actions. However, over time, the actual plant dynamics can drift, leading to a mismatch with the controller's model. This mismatch degrades performance and necessitates re-identification of the plant.
Identifying systems in a closed-loop configuration (where the controller is active during identification) is particularly tricky. The controller's primary goal is to maintain stable operation, which often means suppressing variations in the system's input and output. This inherent stability can lead to a lack of "excitation" – the rich, varied signals needed to accurately estimate the system's dynamics. Injecting extra excitation can disrupt normal operations, impacting product quality or efficiency. This paper addresses the problem of designing excitation signals for closed-loop identification that are both effective for accurate modeling and minimally disruptive to the plant's performance, with a focus on the intended use of the identified model.
Methods
The authors propose an "application-oriented" approach to experiment design. This means the excitation signal is designed not just to make the model accurate in general, but to ensure the model is accurate for the specific control task it will be used for.
Key methodological contributions include:
- Dual Control Formulation: The core idea is to treat the identification process as an optimal control problem. The controller must simultaneously achieve good control performance and ensure the identification experiment yields a sufficiently accurate model for that specific control application.
- Markov Decision Process (MDP) Approach: For a general framework, the problem is formulated as a constrained MDP. This allows for a broad treatment of system dynamics and noise, but can be computationally demanding.
- Model Predictive Control (MPC) Approximations: To make the problem computationally tractable, the authors develop MPC-based controllers. These controllers use a receding horizon approach, optimizing over a finite future horizon.
- MPC-X: This controller directly incorporates the experiment design constraint into the MPC optimization problem. It aims to satisfy the application-specific model accuracy requirement.
- Minimum Time MPC-X: This variant focuses on minimizing the experiment duration while still satisfying the application-specific model accuracy and control performance requirements. It achieves this by adaptively adjusting an excitation scaling factor.
The methods explicitly account for:
- Time-domain constraints: Ensuring that control inputs and outputs remain within operational limits.
- Application cost: Quantifying how much performance degrades if the identified model is not accurate enough for the intended control task.
- Information matrix: This statistical measure quantifies the amount of information about the system parameters that can be extracted from the data. The experiment design aims to maximize this information, specifically for the parameters relevant to the application.
Datasets
Not discussed in the paper. The paper focuses on theoretical development and simulation examples.
Key Insights and Ideas
- Excitation vs. Control: The fundamental trade-off between generating informative signals for identification and maintaining stable, high-performance control is central. The proposed methods offer a principled way to navigate this trade-off.
- Application-Specific Identification: The "aha moment" is realizing that not all model parameters are equally important for a given control task. By focusing excitation on the parameters most relevant to the application, the identification can be more efficient and less disruptive.
- Dual Control for Experiment Design: Framing experiment design as a dual control problem (where the controller acts to both control and learn) is a powerful conceptual shift.
- MPC as a Practical Tool: MPC's receding horizon principle provides a computationally feasible way to implement these complex dual control strategies for experiment design.
Key Results
The paper presents simulation results comparing their proposed methods (MDP and MPC-X) against other approaches, particularly those focusing on "persistence of excitation."
- Improved Model Accuracy for Application: In Example 1, the MDP approach, which explicitly models the noise, resulted in confidence ellipses for parameter estimates that were better aligned with the true system and met the application requirements more reliably than MPC-X (which assumed an output error model).
- Efficiency and Performance: In Example 1, both MPC-X and the MDP approach satisfied the experiment design constraint. The MDP approach showed a slight advantage in meeting the application requirements.
- Minimum Time vs. Regular MPC-X: In Example 2, the minimum time MPC-X achieved the desired information matrix level in significantly fewer samples (82 vs. 162 for PE-MPC with p1=1) and with lower overall cost, demonstrating its efficiency.
- Trade-offs: The simulations highlight that methods focusing on persistence of excitation (like PE-MPC) can lead to higher signal variances and control costs compared to application-oriented methods.
Limitations and Future Work
- Output Error Assumption: The MPC-X controllers rely on the assumption that the system is of output error type (K(0)=0). This is a significant limitation in closed-loop settings and the authors suggest future work should extend these formulations to more general noise models, potentially using stochastic MPC.
- Computational Complexity: While MPC offers a practical solution, the MDP approach can suffer from the "curse of dimensionality" when discretizing continuous state and action spaces. Future research could explore MDPs over infinite spaces or improved discretization techniques.
- Formal Proofs: The authors acknowledge that while simulations show reduced disruption compared to persistence of excitation, formal proofs are lacking and should be investigated further.
- Parameter Estimation: The proposed methods often rely on having a good initial estimate of the system parameters. Future work could explore adaptive input design strategies.
Summary updated