Abstract
This paper introduces a unified framework for controlling stochastic systems that balances cost-minimization with safety by using Conditional Value-at-Risk (CVaR). By reformulating complex risk constraints into a tractable dynamic programming problem, the authors provide a way to design controllers that don't just avoid failure, but specifically limit the severity of potential safety violations.
Introduction
In safety-critical systems—such as autonomous drones or power grids—uncertainty is a fundamental challenge. Traditional "safe" control often relies on probabilistic safe sets, ensuring the system remains safe with a high probability (e.g., 95%). However, these methods possess a significant blind spot: they do not distinguish between a "near miss" and a "catastrophic crash." If a failure occurs, the controller remains indifferent to its severity.
Furthermore, existing methods often decouple safety verification from optimal control, leading to suboptimal performance. This research proposes a "safety-aware" controller that minimizes operational costs while strictly bounding the risk of unsafe states in a computationally feasible manner for real-time applications.
Methodology
The authors propose a risk-based approach utilizing two primary tools:
- Set Distance: "Unsafety" is measured as the distance between the system state and a predefined safe set $A$. Loss is zero within $A$ and grows proportionally with the distance outside it.
- Conditional Value-at-Risk (CVaR): Rather than focusing on the probability of failure (Value-at-Risk), CVaR evaluates the expected value of the loss in the worst-case tail of the distribution (e.g., the worst 5% of outcomes).
Reformulation for Tractability
Standard CVaR constraints lead to "time-inconsistent" problems that are incompatible with standard dynamic programming. The authors prove that the CVaR constraint can be reformulated into a tractable expectation constraint within a Bellman equation. For systems with finitely supported noise, the problem simplifies into a deterministic convex optimization problem, allowing the use of standard convex solvers.
Numerical Evaluation
The approach was evaluated using a Stochastic Inventory Control model. The system manages stock levels under uncertain demand (modeled as a Gaussian distribution $N(20, 6)$). The objective is to minimize holding and lost-sales costs while maintaining inventory within a "safe" range $[0, 100]$. The disturbance distribution was represented using 40 demand samples.
Results and Discussion
The numerical results highlight several key insights:
- Severity Matters: By applying CVaR to the distance from the safe set, the controller is incentivized to remain near the center of the safe region. If it must venture near the edge, it does so in a way that minimizes the "depth" of a potential violation.
- The Risk-Performance Tradeoff: There is a clear "price of safety." As the risk tolerance ($\delta$) decreases (implying tighter safety), the expected minimal cost increases significantly.
- Safe Set Dynamics: The "risk-constrained safe set" emerges naturally from the value function calculation. A state is considered safe if the optimization has a solution ($v_0(x) < \infty$). As risk constraints become stricter, the feasible region of initial states shrinks.
Conclusion
The proposed CVaR-based framework successfully integrates safety constraints into optimal control for stochastic systems. While the reformulation is efficient, the underlying dynamic programming still faces the "curse of dimensionality," limiting its application to low-dimensional systems. Future research will explore occupation measure-based methods for higher-dimensional systems and distributionally robust control to account for uncertainties in noise distributions.
Summary updated