Abstract
The robotics and autonomous systems communities have seen a significant and rapid increase in both the development of robots and vehicles for commercial use and in using these systems across a wide range of novel applications. As these robots, vehicles, software, and even embedded devices move toward much greater autonomy, techniques for verification, providing much higher confidence than usual, are becoming required. However, the analysis and evaluation processes used for traditional systems must be significantly enhanced to provide increased confidence in this next wave of autonomous systems. The need for well understood and effective verification techniques will become even vital, as we move to commercial applications that rely on complex artificial intelligence technologies, and the utilization of these systems in safety-critical scenarios.
Introduction
As autonomous systems (AS) transition from controlled laboratory settings to safety-critical, real-world environments, traditional testing methodologies are proving insufficient. The rapid deployment of autonomous robots and vehicles—ranging from self-driving cars to social robots—has outpaced the development of robust verification frameworks. Unlike traditional systems that often rely on "human-in-the-loop" oversight, modern autonomous systems must make split-second, safety-critical decisions in unconstrained and unpredictable environments.
The IEEE Technical Committee on Verification of Autonomous Systems (TC-VAS) was established to address this gap. By unifying fragmented research efforts, the TC-VAS aims to bridge the divide between rigorous mathematical "formal methods" and practical empirical testing, instilling verification as a core requirement for industry and government standards.
A Taxonomy of Verification Techniques
The TC-VAS advocates for an inclusive approach to verification, categorizing techniques into two primary pillars:
1. Formal Verification (Logic-based)
Formal methods use mathematical logic to provide strong behavioral guarantees.
- Proof: Utilizing mathematical logic to prove that a system ($S$) will always adhere to its requirements ($R$).
- Model Checking: An exhaustive search of every possible execution path within a system model to ensure that no "bad states" (safety violations) are reachable.
- Runtime Verification: Real-time monitoring of a system during operation to ensure it does not violate predefined safety rules.
2. Empirical Verification (Observation-based)
Empirical methods rely on observing system behavior in controlled or real-world settings.
- Software and Simulation Testing: Evaluating the system against a subset of possible scenarios within high-fidelity virtual environments.
- In-situ Testing: Assessing the system's performance and safety in the actual physical world.
As environments become increasingly unstructured, the community is moving from deterministic mathematical proofs toward probabilistic and stochastic models to account for uncertainty.
Challenges and Key Insights
The verification of autonomous systems presents unique challenges that distinguish it from traditional software engineering:
- The Requirement Gap: Defining "desired behavior" ($R$) in a complex, dynamic world is often more difficult than the verification math itself. Completely describing how a robot should act in every possible scenario remains a significant hurdle.
- Defining Autonomy: The TC-VAS defines autonomy not just as "self-driving," but as the ability to make decisions. This ranges from prescripted automation to adaptive systems that evolve based on environmental interactions.
- Unconstrained Environments: In open environments, it is impossible to predict every potential failure mode. Verification must therefore evolve to handle stochastic variables and unconstrained scenarios.
- Interdisciplinary Nature: Verification is not solely an engineering problem; it intersects with psychology (social robotics), philosophy (machine ethics), and law (certification and liability).
Community Impact and Standards
Since its founding in 2019, the TC-VAS has made significant strides in standardizing verification practices:
- Standardization Influence: The committee has provided critical input to major IEEE standards, including IEEE P7009 (Fail-Safe Design of Autonomous Systems) and IEEE P2817 (Guide for Verification of Autonomous Systems).
- Bridging Academia and Industry: The TC has successfully translated academic research into a "summary document" of verification techniques now being integrated into international safety standards.
- Growth: The community has grown to over 490 members, with active monthly webinars fostering knowledge exchange.
Future Directions
While progress has been made, the TC-VAS acknowledges that current research remains fragmented. Future goals for the committee include:
- Roadmapping: Developing a long-term research map for the next decade of autonomous system verification.
- Education: Creating resources to train engineers in specialized verification techniques.
- Open-Source Tools: Establishing a centralized repository for verification tools.
- Dedicated Conferences: Transitioning from workshops to a stand-alone international conference dedicated to the field.
Summary updated