Online Safety Verification for Adaptive Autonomous Systems

37 concepts33 papers

Traditional formal verification operates under a comforting assumption: the system being verified is static. Engineers write code, define an environmental model, and use offline tools like model checking or reachability analysis to prove that the system will never violate its safety specifications.

However, modern autonomous systems—ranging from self-driving cars undergoing online domain adaptation to reinforcement learning controllers managing unmanned surface vehicles—must learn, adapt, and reconfigure themselves in real time. When a system updates its parameters, modifies its internal world models, or dynamically alters its software architecture after deployment, its factory-certified safety guarantees evaporate. As observed by Banerjee et al. and Wagner et al., this creates a critical "verification gap" where a system certified safe at design time can become hazardous as its operational behavior evolves in the real world.

To deploy AI safely in high-stakes environments, the research community is shifting from static, pre-deployment verification to adaptive, online safety verification. This paradigm ensures that safety arguments and control boundaries evolve in tandem with the system's learning and structural changes.


The Core Challenge: The Moving Target of Safety

Why is verifying an adaptive system so difficult? In a static system, the decision boundaries are fixed. In an adaptive system, online learning turns the system's internal logic into a moving target.

Static Verification:  [Fixed Model] + [Fixed Environment] ===> [Permanent Safety Proof]

Adaptive Verification: [Learning Model ➔ Updates] + [Dynamic Environment] ===> [Moving Safety Target]

As highlighted by Jansen et al., traditional formal methods are ill-equipped to handle post-deployment parameter updates. If an autonomous agent rapidly adapts to a novel environment, its original safety specifications can become misaligned with its new context, an open challenge detailed by Delgrange.

This unpredictability complicates regulatory certification across industries like aviation, medicine, and robotics, as noted by Clement et al. and Deshmukh et al.. For instance, Martinsen et al. demonstrate that while online reinforcement learning allows marine vehicles to adapt to unpredictable currents, providing formal guarantees for such real-time parameter adjustments remains a significant hurdle. Furthermore, Botet Colomer points out that without real-time safety monitoring, an online adaptation algorithm might inadvertently learn incorrect or dangerous behaviors from transient, noisy real-world data.


Mechanism 1: Amortizing Computation via Parameterized Safety Zones

The most direct way to verify an adaptive system is to recompute its safe operating envelope whenever it changes. However, formal reachability analysis—calculating the "reachable tube" of states a system might enter—is computationally expensive and usually restricted to offline analysis, a limitation discussed by Lin et al.. A robot cannot afford to pause for hours to re-run a verification proof mid-operation.

To solve this, researchers use deep learning to amortize the computational cost of verification. The key intuition is to treat changing environmental or system parameters not as unpredictable external variables, but as extra dimensions—virtual states—in a higher-dimensional offline calculation.

Traditional Reachability:  Compute Safe Zone = f(Position, Velocity)  --> Must recompute if Mass changes
Parameterized Reachability: Compute Safe Zone = f(Position, Velocity, Mass, Wind Speed)

Borquez et al. pioneered this approach by training neural networks offline to learn parameter-conditioned reachable sets. When the robot encounters hardware degradation (like a failing motor) or shifting external conditions (like wind), it does not re-solve the underlying differential equations. Instead, it instantly queries the pre-computed high-dimensional neural safety zone using its current parameters, updating its safety assurances in real time.

Similarly, Reed et al. leverage Neural Control Lyapunov Functions (CLFs) to synthesize provably stable path-following controllers. By mathematically extending these learned functions from simple straight lines to complex curves, the authors enable autonomous agents to dynamically update and re-verify their control laws over 100 times faster than traditional online optimization methods.


Mechanism 2: Verification-Preserving Model Updates

Another approach focuses on constraining the learning process itself. If we can guarantee that an online model update only occurs when it is mathematically safe, we do not need to re-verify the entire system from scratch.

Fulton et al. introduced this concept through Verification-Preserving Model Updates (VPMUs) within a framework called $\mu$learning. When an autonomous agent learns that its initial environmental model was inaccurate, it transitions to a new model. Crucially, the VPMU framework maintains a set of all feasible models that haven't been falsified by real-world data. The system ensures that every action taken is provably safe under all remaining candidate models, allowing the agent to adapt its behavior based on new data without ever stepping outside the bounds of formal safety.

[Real-World Data] ➔ Falsifies inaccurate models ➔ Narrows down candidate models
                                                        │
                                                        ▼
                                          [Action must be provably safe]
                                          [ across ALL remaining models ]

Where internal model constraints are impossible—such as in complex deep learning architectures—runtime verification can be achieved by monitoring input/output traces. Banerjee et al. demonstrate that by mining "hybrid automata" (mathematical models combining discrete logic with continuous physics) directly from a system's live I/O data, operators can immediately detect when an AI's real-world behavior begins to dangerously diverge from its original safety-certified design. This concept extends even to security contexts; Wang et al. show that dynamic defenses, which alter model parameters during inference to mitigate adversarial attacks, require specialized verification techniques because their decision boundaries shift with every incoming input.


Mechanism 3: Co-evolving Specifications and Dynamic Safety Cases

For systems operating over weeks, months, or years—a challenge known as Long-Term Autonomy (LTA)—the environment, the tasks, and the system structure are all in constant flux, as surveyed by Kunze et al.. In these scenarios, it is not just the controller parameters that change; the software architecture or the system configuration itself might adapt.

For instance, Weyns et al. describe self-evolving computing systems that autonomously modify their own software architecture to handle unexpected anomalies. Similarly, in medical contexts, King et al. address "on-demand" medical cyber-physical systems assembled at a patient's bedside from a diverse pool of interoperable devices. Because the exact combination of components isn't known until the moment of assembly, safety verification must shift from evaluating fixed configurations to verifying dynamically composed systems at the point of care.

To maintain safety under such structural fluidity, the verification framework itself must adapt. Gabor et al. proposed the scenario coevolution paradigm. Instead of testing a system against a static suite of scenarios, the test suite is treated as a competing population that evolves, adapts, and generates increasingly difficult edge cases in parallel with the self-adaptive system. This continuous competition ensures that as the system's state space expands, the verification boundaries expand to cover it, a necessity when dealing with the vast state spaces of dynamically adaptive systems described by Bartel et al..

[Self-Adaptive System] ◄─── Competes & Co-evolves ───► [Dynamic Test Suite]
  (Updates parameters/structure)                         (Generates new edge cases)

Ultimately, this requires a shift in how we structure safety arguments. Lamp et al. argue that safety assurance frameworks designed for frozen models cannot account for systems that employ online learning to adapt to individual users (such as artificial pancreas systems). They advocate for dynamic safety cases—structured arguments that automatically update and re-verify their claims as the underlying machine learning models evolve. This aligns with the broader vision articulated by Srivastava et al., who suggest evaluating AI safety not through static internal performance metrics, but through declarative principles that explicitly measure how safely and transparently a system learns from and interacts with its human operators over time.

Key Foundational Work

Reflective Analysis and Design for Adapting Object Run-Time Behavior (2002) — contributed 2 concept elements to this direction.
Model-Based Approach For Intelligent Control (1988) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_adaptive_formal_verification_ab75d36c,
  title = {adaptive formal verification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adaptive-formal-verification/rdr_ab75d36c598240aa9b40333b92b88275},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_adaptive_formal_verification_ab75d36c_v1,
  title = {adaptive formal verification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adaptive-formal-verification/rdr_ab75d36c598240aa9b40333b92b88275/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake