Overcoming Conservatism in Safety Critical Control

32 concepts31 papers1 proposal

When designing autonomous systems—whether they are self-driving cars, fast-charging batteries, or medical devices—engineers face a fundamental conflict between safety and performance. To guarantee that a system never violates critical physical boundaries, the traditional engineering response is to build in wide safety margins based on worst-case scenarios.

However, this absolute protection comes at a steep price: controller conservatism. When a system is forced to operate under rigid, worst-case assumptions, it becomes trapped far away from its true physical capabilities. An autonomous vehicle might become paralyzed behind a slightly slower car because passing looks marginally risky; a battery charger might take twice as long as necessary to avoid overheating; or an MRI scanner might default to low-resolution imaging because the patient's exact position is uncertain. As noted by Keil et al., defaulting to the most restrictive safety settings invariably sacrifices system utility.

To close this performance gap, a major research direction focuses on mitigating controller conservatism. The core insight is that we can maintain rigorous safety guarantees while dramatically increasing performance by moving away from static, worst-case bounds toward dynamic, data-driven safety envelopes.

The Root of Conservatism: Static and Decoupled Bounds

To understand why traditional systems are overly conservative, we must look at how safety is mathematically enforced. A foundational framework for safety-critical control relies on Control Barrier Functions (CBFs), surveyed extensively by Ames et al.. CBFs act as mathematical guards, defining a "safe set" of states and ensuring the system never leaves it.

The conservatism trap springs from two main design flaws in traditional implementations:

  1. Worst-Case Dynamics: When a system faces model uncertainty or potential adversarial attacks, controllers are typically designed around the absolute worst-case bounds. As Al Maruf et al. point out, designing for the worst-case scenario forces the controller to sacrifice operational efficiency and energy goals to maintain absolute safety. In high-dimensional systems, Tan et al. observe that this reliance on loose worst-case bounds can lead to complete system paralysis.
  2. Decoupled Objectives: Traditional architectures often separate safety verification from the primary control objective. Yang et al. argue that when safety sets are calculated independently of what the system is actually trying to achieve, the resulting policy fails to exploit the full operational range of the system, leading to highly suboptimal behavior.

If the safety parameters are fixed and rigid, the system is forced into a permanent trade-off: be too cautious to complete the task efficiently, or be too aggressive and risk catastrophic failure (Berducci et al.; Cosner et al.).

Shrinking Uncertainty to Expand Performance

The modern solution to the conservatism trap is to shrink the uncertainty envelope online. Instead of designing a static controller for a massive cloud of possible models, modern controllers use data-driven methods to learn the true system dynamics and environmental constraints in real time. As the system becomes more certain about its environment, the safety margins can safely contract, allowing the system to operate right up to its physical limits.

1. Online Mismatch Learning

One way to mitigate conservatism is to learn model inaccuracies on the fly. For instance, Zheng et al. use Gaussian Processes to learn physical model errors online, feeding this updated uncertainty directly into a quadratic program. By reducing the model-plant mismatch dynamically, the controller can maintain high tracking performance without triggering overly conservative safety defaults. Similarly, Holicki et al. demonstrate that systematically refining uncertainty partitions through experimental exploration directly reduces the "price of safety" in control design.

2. Adaptive Parameter Tuning via Optimization

Rather than keeping safety gains fixed, controllers can use optimization techniques to discover the most permissive safe parameters. Hirt et al. and Hirt et al. apply Safe Bayesian Optimization to automatically tune Model Predictive Control (MPC) parameters and constraint backoffs over multiple operational cycles. This allows a battery fast-charging system to safely minimize charging time even when starting with a 50% model mismatch. In dynamic multi-agent environments, Berducci et al. leverage reinforcement learning to dynamically tune CBF parameters, preventing the rigid, overly cautious behaviors that typically emerge from static safety gains.

Decoupling Safety Shields from Performance Optimizers

A powerful architectural paradigm for eliminating conservatism is the complete separation of concerns: decoupling the performance-seeking agent from the safety-guaranteeing mechanism.

Instead of forcing a single neural network or controller to balance both soft performance preferences and non-negotiable safety constraints, developers synthesize a permissive safety shield. The performance agent is free to explore, learn, and act aggressively within this shield. The shield only intervenes when the agent attempts an action that would mathematically guarantee a future safety violation.

┌────────────────────────────────────────────────────────┐
│               Performance Optimizer                    │
│   (e.g., Reinforcement Learning / Neural Network)      │
└──────────────────────────┬─────────────────────────────┘
                           │ Proposed Action
                           ▼
┌────────────────────────────────────────────────────────┐
│                Permissive Safety Shield                │
│   (e.g., Maximally Permissive Control Envelopes / CBF) │
└──────────────────────────┬─────────────────────────────┘
                           │ Validated Safe Action
                           ▼
                  [ Physical System ]

To ensure this shield doesn't re-introduce conservatism, the mathematical boundaries must be as wide as possible. Kabra et al. developed an automated algorithm called CESAR that synthesizes maximally permissive control envelopes by framing the problem as a mathematical game. This ensures that runtime monitors safeguard untrusted, high-performance controllers (like deep neural networks) without unnecessarily diminishing their utility.

Similarly, Wen et al. decouple correct-by-construction temporal logic constraints from reinforcement learning, creating a safe envelope that protects the system during exploration. Belta et al. employ a similar shielding concept using Lyapunov-like Control Barrier Functions to protect model-based reinforcement learning agents before they have fully converged.

By combining rigorous formal methods with real-time data-driven adaptation, this research direction shifts the paradigm from playing it safe through caution to achieving optimal performance through precise, dynamic boundaries.

Key Foundational Work

Control barrier function based quadratic programs with application to adaptive cruise control (2014) — contributed 1 concept elements to this direction.
Patient Position Detection for SAR Optimization in Magnetic Resonance Imaging (2006) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_safety_performance_trade_off_5bb25598,
  title = {safety-performance trade-off},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/safety-performance-trade-off/rdr_5bb255985ba744708d0aa64d2679fc82},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_safety_performance_trade_off_5bb25598_v1,
  title = {safety-performance trade-off},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/safety-performance-trade-off/rdr_5bb255985ba744708d0aa64d2679fc82/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake