Risk-sensitive safety specifications for stochastic systems using Conditional Value-at-Risk

Marco PavonearXiv 2019 · arXiv

TL;DR

This paper introduces a new method for analyzing the safety of stochastic systems by using Conditional Value-at-Risk (CVaR) to balance worst-case and risk-neutral perspectives. This approach allows for more nuanced safety specifications, especially for systems with rare but potentially catastrophic failure modes, and provides a tractable algorithm for synthesizing safety-critical policies.

What Problem Does This Paper Address?

Traditional safety analysis for stochastic systems often relies on expected costs, which can overlook the impact of rare, high-consequence events. While robust control methods address worst-case scenarios, they can be overly conservative. This paper bridges this gap by proposing a risk-sensitive safety specification that uses CVaR to explicitly account for the "fat tails" of cost distributions, allowing for a tunable trade-off between worst-case and risk-neutral behavior. This is crucial for safety-critical systems where minimizing the probability of extreme failures is paramount.

Methods

The authors introduce a "risk-sensitive safe set" defined by the sublevel set of an optimal control problem using the CVaR measure. Since this problem doesn't satisfy the standard Bellman's Principle, they propose an under-approximation using a CVaR-Markov Decision Process (CVaR-MDP). A key contribution is the development and proof of correctness for a value iteration algorithm that solves this reduced CVaR-MDP problem, enabling tractable policy synthesis for linear systems. They also extend this to non-linear systems through a realistic numerical example.

Datasets

Not discussed in the paper. The paper uses simulated data from a stormwater system and a thermostatically controlled load system for evaluation.

Key Insights and Ideas

  • CVaR for Safety: CVaR offers a more interpretable and direct way to penalize tail risk compared to traditional methods like exponential disutility, which primarily focus on mean and variance.
  • Tunable Risk Sensitivity: The CVaR approach allows for a flexible balance between worst-case and risk-neutral perspectives, controlled by a risk-sensitivity parameter.
  • Tractable Algorithms: The development of a value iteration algorithm for CVaR-MDPs makes risk-sensitive safety analysis computationally feasible, especially for linear systems.
  • Pre-commitment Policies: The paper shows that by committing to specific risk-sensitivity level dynamics, history-dependent policies can be summarized into more manageable "X-Markov" policies, simplifying synthesis.
  • Non-linear System Applicability: The stormwater system example demonstrates that the proposed framework can be applied to non-linear systems, extending its practical relevance.

Key Results

  • The paper formally proves the correctness of a value iteration algorithm for computing risk-sensitive safe sets for linear systems.
  • A realistic two-tank stormwater system example demonstrates the approach's applicability to non-linear systems.
  • Numerical comparisons with the exponential disutility criterion show that CVaR can achieve comparable or better risk reduction in tail events, especially when control authority is limited or cost distributions are asymmetric.
  • The computational tractability of the algorithm is demonstrated, with an example taking approximately 230 hours on a 4-core machine for a moderately complex system.

Limitations and Future Work

  • The formal proofs of the value iteration algorithm and policy synthesis are primarily for linear time-varying (LTV) systems, although the approach is demonstrated on a non-linear system.
  • The paper notes that while CVaR provides a direct way to minimize tail risk, its implementation and theoretical underpinnings are more involved than standard methods.
  • Future work includes extending the approach to a larger class of systems, leveraging novel approximate dynamic programming techniques, and exploring transformations between non-linear and linear dynamics.

Summary updated

Abstract

This paper proposes a safety analysis method that facilitates a tunable balance between the worst-case and risk-neutral perspectives. First, we define a risk-sensitive safe set to specify the degree of safety attained by a stochastic system. This set is defined as a sublevel set of the solution to an optimal control problem that is expressed using the Conditional Value-at-Risk (CVaR) measure. This problem does not satisfy Bellman's Principle, thus our next contribution is to show how risk-sensitive safe sets can be under-approximated by the solution to a CVaR-Markov Decision Process. We adopt an existing value iteration algorithm to find an approximate solution to the reduced problem for a class of linear systems. Then, we develop a realistic numerical example of a stormwater system to show that this approach can be applied to non-linear systems. Finally, we compare the CVaR criterion to the exponential disutility criterion. The latter allocates control effort evenly across the cost distribution to reduce variance, while the CVaR criterion focuses control effort on a given worst-case quantile--where it matters most for safety.
BibTeX
@misc{pavone2019risk,
  author = {Marco Pavone},
  title = {{Risk-sensitive safety specifications for stochastic systems using Conditional Value-at-Risk}},
  year = {2019},
  eprint = {1909.09703},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.09703},
}
Figure