Abstract
In high-stress aviation environments, pilots require immediate and contextually accurate guidance. While Large Language Models (LLMs) offer conversational flexibility, their propensity for hallucinations and lack of formal reasoning make them unsuitable for safety-critical decision-making. This paper presents a hybrid neuro-symbolic dialogue system that combines the linguistic capabilities of BERT-based Natural Language Understanding (NLU) with the logical rigour of Fully Observable Non-Deterministic (FOND) and Hierarchical Task Network (HTN) planning. By modeling dialogue as a state-space problem, the system ensures verifiability and goal-directed assistance. To address the scarcity of emergency-specific voice data, we employ LLM-based data augmentation. Our results demonstrate that this hybrid approach maintains a 100% success rate in complex scenarios where pure LLM performance degrades significantly, providing a robust framework for human-in-the-loop assistance in cockpit environments.
Introduction
In the high-stakes environment of an aircraft cockpit, pilots must manage complex systems while responding to rare, life-threatening emergencies. Effective assistance requires a "human-in-the-loop" system capable of providing contextual guidance based on both sensor data and pilot observations (e.g., smoke color or fire location). Current AI solutions face two primary hurdles: the "Reasoning Gap," where Large Language Models (LLMs) struggle with logical consistency and safety-critical verifiability, and the "Data Scarcity Gap," resulting from the lack of real-world audio recordings from actual aviation emergencies.
System Architecture
To overcome these challenges, we propose a neuro-symbolic framework that integrates deep learning for natural language processing with symbolic planning for decision logic. The system architecture, illustrated in Figure 1, comprises several key components:
- Natural Language Understanding (NLU): A BERT-based model extracts intents and entities from pilot speech.
- Dialogue Executor (FOND Planning): Uses Fully Observable Non-Deterministic planning to manage the conversation. By treating dialogue as a state-space problem, the system generates a contingent plan that accounts for various pilot responses.
- Instructions Generator (HTN Planning): Employs Hierarchical Task Networks to produce step-by-step technical instructions once the emergency context is confirmed.
Verifiable Dialogue Management
A core advantage of our approach is verifiability. Unlike "black box" LLMs, the FOND-based dialogue manager operates on a formal plan. This allows developers to mathematically guarantee that the system will reach a goal state—providing the correct emergency procedure—provided the pilot supplies the necessary information. Figure 2 visualizes the dialogue state graph, showing the various paths the conversation can take to resolve specific emergency contexts. This formal structure prevents the "hallucinations" common in pure neural architectures.
Data Augmentation and Evaluation
To address the lack of training data, we utilized GPT-4 to generate a synthetic aviation dataset based on a small sample of real pilot surveys. This "teacher" model approach allowed us to train a robust NLU system that achieved an 89% intent classification accuracy and a 90% F1 score for entity extraction.
Experimental evaluations in the Ultralight (UL) aviation domain revealed a significant "reasoning degradation" in pure LLMs. While GPT-3.5 and GPT-4 success rates plummeted to 14% and 42% respectively in complex fire scenarios, our hybrid system maintained a 100% success rate. This underscores the necessity of symbolic planning for maintaining reliability as task complexity increases.
Conclusion
By blending the linguistic flexibility of neural networks with the logical rigor of symbolic planners, we have developed a dialogue system capable of assisting pilots during critical flight phases. Future work will focus on handling mid-conversation goal shifts and testing performance under the acoustic stress of a real cockpit environment.
Summary updated