TL;DR
Standard explainable AI (XAI) methods often generate "impossible" network states to explain model predictions, leading to unreliable results in security contexts. This paper introduces Protocol-Aware XAI (PA-XAI), a framework that enforces domain-specific rules on explanation queries, revealing that many "standard" explanations are artifacts of domain blindness rather than true model insights.
What Problem Does This Paper Address?
Feature attribution methods like LIME, SHAP, and Integrated Gradients work by querying a model with perturbed versions of an input to see how the output changes. In Network Intrusion Detection Systems (NIDS), however, features are governed by strict protocols. For example, a packet cannot have both a zero duration and a high byte count, nor can it have TCP-specific flags if the protocol is UDP.
When an XAI tool generates a "rule-violating" state to probe a model, it suffers from domain blindness. The model is forced to make a prediction on data it would never see in operation. Consequently, the resulting explanation might reflect the model's arbitrary behavior on "garbage" data rather than its actual logic for legitimate traffic. This is a critical failure for security analysts who rely on these explanations for forensic investigation and threat hunting.
Methods
The authors formalize this problem and propose a solution: Protocol-Aware XAI (PA-XAI). Instead of allowing explainers to query any point in the feature space, PA-XAI constrains them to a "rule-valid set" ($V_R$).
Key components include:
- Valid-Query Invariance Theorem: A formal proof showing that if two models agree on all valid inputs, an XAI procedure restricted to those inputs will yield identical attributions for both models.
- Protocol-Aware Adaptations: The authors "repair" four popular XAI families:
- PA-LIME: Uses protocol-conditioned sampling and a repair routine ($\Pi_R$) to ensure neighborhood points are valid.
- PA-SHAP: Restricts background references to benign, rule-valid records.
- PA-IG & PA-DeepLIFT: Control path and reference selection to avoid fractional discrete interpolations (e.g., a "half-TCP" packet).
- Multi-Module Audit: A six-coordinate profile $(F, S, R, C, U, T)$ that evaluates model alignment (Faithfulness), stability, adversarial robustness (Stress), cross-method agreement, and feature-reduction utility.
Datasets
The framework was evaluated on four widely used tabular NIDS benchmarks:
- NSL-KDD
- CIC-IDS-2017
- UNSW-NB15
- CSE-CIC-IDS2018
These datasets provide a mix of continuous and discrete features with complex inter-dependencies (e.g., protocol-gated flags) ideal for testing domain constraints.
Key Insights and Ideas
- Repeatability can be a lie: In several cases, standard LIME reported "perfect" repeatability (Jaccard = 1.0). However, PA-LIME revealed that this was an artifact of querying invalid space; once restricted to valid states, the mean repeatability dropped to 0.575, exposing the true sensitivity of the neighborhood construction.
- Faithfulness is fragile: The "faithfulness" of an explanation (how well it tracks model score changes when features are removed) actually changed sign in 5 out of 12 LIME comparisons. This means a standard audit might conclude a feature is important when it's actually inversely associated with the model's logic in the valid domain.
- Gradient methods are more robust: Under adversarial stress (using FGSM attacks followed by protocol repair), PA-DeepLIFT and PA-IG maintained the highest directional alignment with original explanations, outperforming perturbation-based methods like LIME and SHAP.
Key Results
- Stability Shift: Enforcing protocol awareness caused a significant drop in LIME's reported stability, from a "perfect" 1.0 to a mean of 0.575.
- Adversarial Robustness: PA-DeepLIFT and PA-IG achieved the best attribution-similarity scores under attack, with curve areas of 0.452 and 0.413, respectively.
- Pareto Frontier: In feature reduction tasks, PA-SHAP-DNN and PA-IG-DNN defined the "nondominated" frontier, offering the best trade-off between aggressive data compression and maintaining model performance (weighted-F1).
Limitations and Future Work
The authors acknowledge several areas for expansion:
- Encoded vs. Raw: The current audit focuses on encoded feature space. Future work should bridge the gap to "raw-flow reconstruction," ensuring explanations are valid at the packet level.
- Adaptive Attacks: While they tested standard adversarial attacks, "explanation-targeting" attacks that specifically try to fool the PA-XAI constraints remain an open challenge.
- Human-in-the-loop: Ultimately, the "trustworthiness" of an explanation is a human judgment. The authors suggest that future studies should evaluate how PA-XAI affects analyst decision quality and workflow efficiency.
This work serves as a wake-up call for XAI in structured domains: an explanation is only as credible as the queries used to generate it.
Summary updated