Abstract
Designing controllers for systems with stochastic noise typically focuses on minimizing expected costs. However, in safety-critical applications, "tail risks"—rare but catastrophic events—must be managed. This paper introduces a framework for risk-sensitive affine control of linear time-invariant (LTI) systems subject to nonzero-mean noise. By optimizing the Worst-case Conditional Value-at-Risk (W-CVaR) of the stationary distribution, the authors address the critical coupling between control gains and bias terms. We present an alternating optimization algorithm that ensures convergence and demonstrates superior performance in robotics and HVAC benchmarks.
Introduction
Classical control theories, such as the Linear Quadratic Regulator (LQR), are designed to optimize average-case performance. While effective for many applications, these methods often fail to account for the extreme deviations caused by stochastic noise in the system's stationary distribution. In domains like spacecraft docking or climate control, where noise often possesses a constant "drift" or nonzero mean, the stakes for ignoring these tail risks are high. This paper proposes a robust approach to minimize risk while accounting for the complex interactions between system parameters and noise characteristics.
The Challenge of Nonzero-Mean Noise
A significant gap in existing risk-sensitive control research is the assumption of zero-mean noise. This assumption simplifies the problem, allowing engineers to solve for the control gain ($K$) and the bias term ($\ell$) independently. However, the authors demonstrate that when noise has a nonzero mean, these terms become strongly coupled.
Attempting to solve for them separately—a "naive" approach—leads to significant performance degradation as noise intensity or drift increases. The interaction between the state mean and covariance, represented by the off-diagonal elements in the system's moment matrices, must be explicitly managed to ensure safety and efficiency.
Methodology: W-CVaR and BMI
The researchers utilize Worst-case Conditional Value-at-Risk (W-CVaR) as their primary risk metric. Unlike standard CVaR, which requires precise knowledge of the noise distribution, W-CVaR only requires the first two moments (mean and covariance). This makes the resulting controller robust against model uncertainty.
The optimization problem is naturally non-convex. To address this, the authors reformulate it into a Bilinear Matrix Inequality (BMI) using Semi-Definite Programming (SDP) relaxations. They propose an Alternating Optimization Algorithm that iteratively solves two convex sub-problems:
- Step I: Fix the controller parameters and optimize the risk measures.
- Step II: Fix the risk measures and optimize the controller parameters.
This sequence is mathematically proven to be monotonically decreasing and guaranteed to converge to a stable solution.
Key Insights and Numerical Results
The core insight of the paper is the necessity of joint optimization. As the noise mean ($\mu$) or variance grows, the gap between the proposed risk-sensitive approach and standard LQR baselines widens.
As illustrated in Figure 4, the optimal risk-sensitive parameters ($k^*, \ell^*$) deviate significantly from LQR values to minimize the W-CVaR. The coupling term $p^*$ tracks the interaction between the mean and covariance, confirming that risk-sensitive control must account for these dynamics to protect against extreme events.
Benchmarks: HVAC and Robotics
The algorithm was evaluated on two simulated benchmarks: an Inverted Pendulum and a Multi-zone HVAC Thermal Control system.
In the Inverted Pendulum tests, the risk-sensitive controller consistently achieved lower W-CVaR values than LQR, particularly as the system's instability increased. In the HVAC simulation—a high-dimensional system where thermal dynamics involve constant heat sources acting as nonzero-mean noise—the controller significantly reduced worst-case temperature fluctuations.
Crucially, as shown in Figure 6, this risk reduction comes with almost zero penalty to the average LQR performance. The algorithm's fast convergence (typically within 500 iterations) makes it a viable candidate for medium-scale engineering problems.
Conclusion
This work provides a robust framework for designing risk-sensitive controllers in the presence of biased noise. By addressing the coupling between gain and bias through a BMI reformulation and an efficient alternating optimization algorithm, the authors offer a practical solution for safety-critical LTI systems. Future work will explore scalability to higher-dimensional systems and decentralized control architectures.
Summary updated