This article summarizes the proceedings of the 15th NASA Formal Methods Symposium (NFM 2023). The collection of 29 peer-reviewed papers and three invited talks represents the current state of "formal methods"—the application of mathematical logic and rigorous proofs to ensure software and hardware reliability in mission-critical environments.
The Challenge: Assurance at Scale
As NASA advances the Artemis program and develops autonomous deep-space habitats like the Gateway lunar station, systems are reaching a level of complexity where traditional "test-and-fix" cycles are no longer sufficient. In deep space, a software bug in a life-support controller or a navigation error in a self-driving rover can have catastrophic consequences.
The symposium specifically addresses three core challenges in modern aerospace engineering:
- The AI "Black Box": Developing methods to trust neural networks in flight controllers and collision avoidance systems.
- Autonomous Complexity: Verifying multi-agent systems that must coordinate without human intervention.
- The Specification Gap: Translating high-level safety requirements into precise mathematical logic.
Methodological Innovations
The research presented at NFM 2023 highlights a diverse toolkit of rigorous approaches designed to handle these complexities:
- Neural Network Verification: Researchers are utilizing "Polynomial Zonotopes" and tools like "Marabou" to calculate reachable states, proving that AI models will remain within safe boundaries regardless of input noise.
- Assume-Guarantee Contracts: To manage massive systems like the Gateway, engineers break them into smaller components. Each component operates under a "contract": if inputs meet specific criteria, the component guarantees a safe output.
- Advanced SMT Solving: New algorithms for Satisfiability Modulo Theories (SMT) allow for more efficient searches for counter-examples, mathematically "breaking" a system to identify hidden bugs before deployment.
- Correct-by-Design Synthesis: Moving beyond post-hoc verification, some methods now automatically generate code from formal specifications, ensuring inherent correctness.
Key Insights and Industry Impact
Several themes emerged from the symposium that signal a shift in how aerospace software is developed:
- Verifiable AI: Machine learning is no longer viewed as inherently unpredictable. By applying formal constraints to activation functions (such as LSTMs), researchers are demonstrating that "Verified AI" is a practical, reachable goal.
- "Shifting Left" in Design: Formal methods are increasingly used during the requirement phase rather than just for post-coding verification. This allows engineers to catch logic flaws before a single line of code is written.
- Hybrid and Stochastic Systems: There is a growing focus on systems that combine discrete computer logic with continuous physical movement, accounting for real-world noise and uncertainty through stochastic dynamical models.
Results and Tooling
The symposium maintained a high standard of rigor with a 38% acceptance rate. Notable practical applications discussed include the Vehicle Systems Manager (VSM) for the Gateway lunar station, which uses formal contracts for mission and fault management. Additionally, the release of open-source toolkits like VerifAI and Scenic provides the broader engineering community with the means to design and verify AI-based autonomy in cyber-physical systems.
Limitations and Future Outlook
Despite significant progress, the "state-space explosion" remains a hurdle; verifying massive, interconnected "systems-of-systems" is still computationally intensive. Furthermore, modeling the interaction between formal autonomous systems and unpredictable human operators remains a complex frontier. Future work will likely focus on improving heuristics for non-linear transcendental arithmetic—essential for the physics of orbital mechanics—and scaling proofs to meet the demands of the next generation of space exploration.
Summary updated