Introduction
As artificial intelligence (AI) models in training simulations grow in complexity, they often become "black boxes" to the users they are intended to teach. In military After-Action Reviews (AARs) or social skill training, students must understand the rationale behind an AI’s actions to learn effectively. However, most existing explanation systems are hard-coded into specific simulators, making them impossible to port to new domains. This paper presents a modular, domain-independent architecture for Explainable AI (XAI) designed to provide human-understandable justifications for simulated behavior across diverse environments.
A Modular XAI Architecture
The core of the proposed system is a multi-layered architecture that separates data storage, reasoning, and communication. By decoupling the explanation logic from the underlying simulation code, the framework can interface with various AI systems.
The architecture consists of several key components:
- Data Integration: Simulation logs and scenario data are imported into a relational database to handle the massive data scales of tactical simulations.
- Behavior Translation: Simulation-specific behaviors (procedural or rule-based) are mapped into a common representation of goals, preconditions, and effects.
- Logical Forms (LF): Instead of hard-coding questions, the system uses abstract representations of questions. This allows for the dynamic generation of natural language and database queries.
- Dialogue Management & NLG: A dedicated manager handles user interaction, using XSL templates for Natural Language Generation (NLG) to turn database results into conversational English.
- Question Unlocking: To prevent overwhelming the user, the system "unlocks" deeper "why" questions only after foundational "what" questions have been addressed, creating a guided investigation path.
Cross-Domain Evaluation
The architecture was evaluated across two distinct simulation environments: the OOS (One Semi-Automated Forces Objective System), a high-fidelity tactical military simulation, and Virtual Humans, a social simulation for teaching negotiation and cultural awareness.
In the Virtual Humans domain, the system supported 110 distinct questions generated dynamically from logical forms. It successfully demonstrated traceable reasoning by linking high-level outcomes, such as a failed negotiation, to specific simulation variables like a decrease in "solidarity" caused by a student's specific dialogue choice.
Key Insights: Intent and the Teachable Moment
A primary finding is that for AI to be explainable, it must be built with "goal-based" or "rule-based" representations. Purely procedural code lacks the explicit representation of intent necessary for meaningful justification.
Furthermore, the XAI system acts as more than a passive Q&A tool. It integrates with an automated "Tutor" component that identifies "teachable moments"—critical events like a sudden drop in a character's trust—and prompts the student to use the XAI system to investigate their own mistakes.
Conclusion and Future Directions
The authors successfully demonstrated that a single "reasoner" and "NLG" component could be reused across vastly different domains, from combat tactics to social negotiation. While the system automates much of the process, identifying specific "teaching points" still requires some manual annotation by experts. Future work aims to further automate the mapping between question logical forms and query logical forms, reducing the effort required to deploy XAI in new simulations.
Summary updated