Covariance Steering With Optimal Risk Allocation

Joshua Pilipovsky, Panagiotis Tsiotras

This paper presents a significant advancement in stochastic control by moving away from "one-size-fits-all" safety margins. By treating risk as a resource to be optimized rather than a constant to be distributed evenly, the authors provide a framework for more efficient, less conservative autonomous systems—specifically demonstrated through the high-stakes lens of orbital mechanics.

TL;DR

The authors develop an Iterative Risk Allocation (IRA) algorithm for "Covariance Steering," allowing a system to steer its entire probability distribution toward a goal while intelligently spending a "risk budget." By optimizing how risk is distributed across different constraints and time steps, the method achieves lower control costs and less conservative trajectories than traditional uniform-risk approaches.

What Problem Does This Paper Address?

In stochastic control, we deal with systems influenced by random noise (like wind gusts on a drone or sensor noise on a spacecraft). Because Gaussian noise is theoretically unbounded, we cannot guarantee 100% safety. Instead, we use chance constraints, which ensure the probability of failure stays below a threshold (e.g., "there is a 97% chance we stay inside this corridor").

The "standard" way to handle multiple constraints over time is to divide the total risk budget ($\Delta$) equally. If you have 10 steps and a 3% total risk budget, you might allow a 0.3% risk at each step. However, this is often inefficient. Some parts of a maneuver are naturally "easier" or "safer" than others. Forcing a uniform risk level everywhere makes the controller overly cautious (conservative), leading to higher fuel consumption or restricted movement.

Furthermore, many real-world constraints are conical (like a camera's field of view or a docking port's approach zone). These are mathematically difficult to handle in a way that remains "convex" (efficiently solvable).

Methods

The paper proposes a two-stage Iterative Risk Allocation (IRA) framework:

  1. Upper Stage (Risk Optimizer): This stage treats the risk levels ($\delta$) for each constraint as decision variables. It uses a convex optimization approach to adjust these levels based on the current controller's performance.
  2. Lower Stage (Controller Optimizer): Given the risk levels from the upper stage, this stage solves a Semi-Definite Program (SDP) to find the optimal control gains that steer the state mean and covariance to the target.

The process repeats until it finds the "sweet spot" where the risk is allocated to the constraints where it provides the most benefit to the total cost.

Handling Cones: The authors introduce three methods to approximate non-convex cone chance constraints as convex Second-Order Cone (SOC) constraints:

  • Three-cut Outer Approximation: A linear relaxation.
  • Reverse Union Bound: A probabilistic approach.
  • Geometric Approximation: A novel 3D-specific method that uses the properties of Gaussian distributions to create a "disk" of safety, which proved to be the least conservative of the three.

Datasets

The researchers evaluated their algorithm using a high-fidelity simulation of a Spacecraft Rendezvous and Docking mission.

  • Dynamics: They used the Clohessy-Wiltshire-Hill equations, which describe the relative motion of two satellites in orbit.
  • Constraints: The "Deputy" spacecraft had to stay within a Line-of-Sight (LOS) cone of the "Chief" spacecraft while dealing with hard thruster saturation limits.

Key Insights and Ideas

  • Risk as a Resource: The core "aha" moment is that risk is a budget. By "spending" more risk on a difficult constraint, you can "save" significantly on control effort (fuel).
  • Maximizing Terminal Covariance: A less conservative solution doesn't just mean "closer to the edge"; it means the controller doesn't have to squeeze the state distribution as tightly. This results in a larger terminal covariance, which is often desirable as it requires less aggressive (and expensive) control.
  • IRA vs. SMPC: Standard Stochastic Model Predictive Control (SMPC) often struggles with "recursive feasibility" and doesn't allow for precise steering of the final covariance. This method handles both, making it better suited for precision tasks like docking.

Key Results

  • Efficiency: In the spacecraft simulation, the IRA method utilized nearly the entire 3% risk budget (achieving a true risk of ~2.99%), whereas the uniform allocation method was "too safe," only reaching ~1.23% risk.
  • Cost Reduction: Because the IRA method wasn't fighting to stay unnecessarily far from the boundaries, it achieved a lower optimal cost.
  • Volume Comparison: The terminal volume (uncertainty) of the IRA solution was significantly larger than the uniform solution. This confirms that the uniform approach was wasting energy to over-constrain the spacecraft's position.
  • Geometric Superiority: The proposed Geometric Approximation for cone constraints outperformed the other two approximations, providing the highest terminal covariance while maintaining safety.

Limitations and Future Work

  • Linearity: The current framework is designed for Linear Time-Varying (LTV) systems. While many systems can be linearized, truly nonlinear dynamics (like high-angle-of-attack flight) would require further extensions.
  • Gaussianity: The math relies heavily on Gaussian noise assumptions. While the authors noted the system handled input saturation (which makes the state non-Gaussian) reasonably well, formal proofs for non-Gaussian distributions are a potential area for future research.
  • Computational Overhead: While more efficient than particle-based methods, the iterative nature of solving multiple SDPs is more computationally intensive than a single-shot solve. Scaling this to very high-dimensional systems or very long time horizons remains a challenge.

Summary updated

Abstract

This article extends the optimal covariance steering (CS) problem for linear stochastic systems subject to chance constraints so as to account for an optimal allocation of the risk. Previous works have assumed a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">uniform</i> risk allocation in order to cast the optimal control problem as a semidefinite program, which can be solved efficiently using standard SDP solvers. An iterative risk allocation (IRA) formalism is used to solve the optimal risk allocation problem for CS using a two-stage approach. The upper-stage of IRA optimizes the risk, which is a convex problem, while the lower-stage optimizes the controller with the new constraints. The process is applied iteratively until the optimal risk allocation that achieves the lowest total cost is obtained. The proposed framework results in solutions that tend to maximize the terminal covariance, while still satisfying the chance constraints, thus leading to less conservative solutions than previous methodologies. In this article, we consider both polyhedral and cone state chance constraints. Finally, we demonstrate the approach to a spacecraft rendezvous problem and compare the results with other competing approaches.
BibTeX
@article{pilipovsky2021covariance,
  author = {Joshua Pilipovsky and Panagiotis Tsiotras},
  title = {{Covariance Steering With Optimal Risk Allocation}},
  journal = {IEEE Trans. Aerosp. Electron. Syst},
  year = {2021},
  doi = {10.1109/taes.2021.3086956},
}

Loading related content...

Figure