Conformal Reachability for Safe Control in Unknown Environments

Xinhang Ma, Junlin Wu, Yiannis Kantaros, Yevgeniy Vorobeychik

TL;DR

Researchers have developed ReCORS, a framework that provides rigorous probabilistic safety guarantees for autonomous systems operating in environments with unknown, stochastic dynamics. By combining Conformal Prediction (to quantify model uncertainty) with Reachability Analysis (to verify safe paths), they’ve created a differentiable pipeline that trains robots to be both high-performing and provably safe.

What Problem Does This Paper Address?

In the world of autonomous systems—like self-driving cars or delivery drones—safety isn't just a "nice-to-have"; it’s a requirement. Most Reinforcement Learning (RL) agents learn to be safe "on average" by receiving penalties for crashes. However, "average safety" isn't enough when a single failure is catastrophic.

Existing methods for provable safety usually fall into two traps:

  1. They assume we already know the exact physics (dynamics) of the world.
  2. They assume the world is deterministic or follows very specific mathematical rules (like being "Lipschitz continuous" or having Gaussian noise).

In reality, dynamics are often unknown, non-linear, and messy. If your model of the world is slightly wrong, your safety "proof" falls apart. This paper addresses the gap of providing distribution-free safety guarantees—meaning they work regardless of the underlying noise distribution—for systems with unknown dynamics.

Methods

The authors introduce ReCORS (Reachability with COnformal uncertainty sets for Reinforcement learning with Safety guarantees). The approach rests on three pillars:

  • Conformal Safety Analysis (CSA): The "secret sauce" is Conformal Prediction (CP). CP is a statistical tool that generates uncertainty intervals (error bars) around a model's prediction. Crucially, these intervals are guaranteed to contain the true value with a user-defined probability (e.g., 95%), without needing to know if the noise is Gaussian or otherwise. ReCORS uses these intervals to perform reachability analysis, checking if the "worst-case" predicted state remains within the safe zone.
  • Safety-Weighted Dynamics: Not all prediction errors are equal. Missing a drone's position by 5cm in an open field is fine; missing it by 5cm near a wall is a crash. The authors use a specialized loss function that penalizes the dynamics model more heavily for errors in "safety-critical" dimensions.
  • Differentiable Pipeline: Standard CP is not "differentiable," meaning you can't easily use it to train a neural network via gradient descent. The authors implemented a differentiable proxy for the conformal bounds, allowing the safety requirement to be baked directly into the RL training objective.

Datasets

The framework was evaluated across seven safe control settings spanning four distinct domains:

  • Cartpole: Balancing a pole while staying within position limits.
  • Lane Following: Keeping a vehicle within a lane.
  • 2D & 3D Quadrotor Control: Navigating a drone to a target while avoiding obstacles (using both linear and non-linear safety specs).
  • Safety Gymnasium (CarGoal & HalfCheetah): High-dimensional benchmarks (up to 72 dimensions) involving complex navigation and locomotion tasks.

Key Insights and Ideas

  • The "Safety-Performance" Tradeoff: Most safe RL agents either play it too safe (staying still to avoid risk) or take too many risks for high rewards. ReCORS allows for a fine-grained tradeoff by maximizing the "planning horizon"—the distance into the future the agent can prove it is safe.
  • Curriculum Learning for Safety: The algorithm uses a curriculum approach. It starts by trying to prove safety for a very short window (e.g., 5 steps ahead). Once it masters that, it incrementally increases the horizon, eventually learning to be safe over long trajectories.
  • No Smoothness Required: Unlike many formal methods that require the system's physics to be "smooth" (Lipschitz continuous), ReCORS makes no such assumptions. This makes it much more robust for complex, neural-network-based controllers.

Key Results

ReCORS was compared against five state-of-the-art baselines (including PPOLag and PCPO).

  • Stronger Guarantees: In the Cartpole environment, baselines couldn't provide safety guarantees beyond a 20-step horizon. ReCORS maintained high-probability safety guarantees for over 50 steps.
  • Higher Utility: In drone experiments, some baselines "cheated" to stay safe by simply hovering in place (earning near-zero reward). ReCORS achieved significantly higher rewards while maintaining the same (or better) verified safety levels.
  • Empirical Robustness: Even over long horizons that weren't explicitly "verified," ReCORS policies resulted in far fewer safety violations than any other method.

Limitations and Future Work

The authors identify one major hurdle: Perfect Observability. The current framework assumes the agent knows its exact state (position, velocity, etc.) at all times. In the real world, sensors are noisy. Future work will likely focus on extending these conformal guarantees to "Partially Observable" environments where the agent only has a fuzzy idea of where it is. Additionally, while the reachability analysis is efficient for the tested dimensions, scaling to even higher-dimensional "pixel-to-control" (vision-based) tasks remains a challenge.

Summary updated

Abstract

Designing provably safe control is a core problem in trustworthy autonomy. However, most prior work in this regard assumes either that the system dynamics are known or deterministic, or that the state and action space are finite, significantly limiting application scope. We address this limitation by developing a probabilistic verification framework for unknown dynamical systems which combines conformal prediction with reachability analysis. In particular, we use conformal prediction to obtain valid uncertainty intervals for the unknown dynamics at each time step, with reachability then verifying whether safety is maintained within the conformal uncertainty bounds. Next, we develop an algorithmic approach for training control policies that optimize nominal reward while also maximizing the planning horizon with sound probabilistic safety guarantees. We evaluate the proposed approach in seven safe control settings spanning four domains -- cartpole, lane following, drone control, and safe navigation -- for both affine and nonlinear safety specifications. Our experiments show that the policies we learn achieve the strongest provable safety guarantees while still maintaining high average reward.
BibTeX
@misc{ma2026conformal,
  author = {Xinhang Ma and Junlin Wu and Yiannis Kantaros and Yevgeniy Vorobeychik},
  title = {{Conformal Reachability for Safe Control in Unknown Environments}},
  year = {2026},
  eprint = {2602.03799},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.03799},
}
Figure