Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody

Jose Luis Vela Alonso, Carmen PellicerarXiv 2026 · arXiv

This paper presents MODEXRE (Modular Digital Examination and Reporting Engine), a framework designed to move network intrusion detection from the "clean" environment of a research lab into the messy, high-stakes world of digital forensics. By combining explainable AI (XAI) with a cryptographically verifiable chain of custody, the authors demonstrate how to build a system that isn't just accurate, but also legally admissible.

TL;DR

MODEXRE is a forensic pipeline that bridges the gap between laboratory AI models and real-world evidence by using a dual-mode architecture (Laboratory vs. Formal) and a hash-chained digital chain of custody. The paper highlights how "perfect" benchmark models often fail in the field due to feature gaps and hidden bugs, and provides a roadmap for building AI systems that are transparent enough for the courtroom.

What Problem Does This Paper Address?

Most research in AI-based intrusion detection focuses on maximizing accuracy on curated datasets like CICIDS2017. However, these models often fail when deployed for actual forensic work. The authors identify three major hurdles:

  1. The Feature Gap: Laboratory datasets often have 70+ features, but real-world evidence (like firewall logs) might only provide 4 or 5.
  2. Admissibility: In a legal context, you must prove that the model used in a case is exactly the same one that was validated in the lab, and you must be able to explain why it flagged a specific event.
  3. Architectural Blindness: Standard event-by-event classifiers often miss attacks that only appear as patterns over time, such as port scanning.

Methods

The authors introduce a dual-mode architecture:

  • Laboratory Mode: Where models are trained, tuned, and certified using synthetic and public data.
  • Formal Mode: A "frozen" environment where a hash-verified model performs inference on real case evidence. No retraining is allowed here, ensuring the model's integrity.

Key Technical Components: * OCSF Normalization: Uses the Open Cybersecurity Schema Framework to translate heterogeneous logs (Suricata, CEF, PCAP) into a common format. * Causal Temporal Flow Aggregation: A layer that looks at windows of time (e.g., 60 seconds) to calculate features like "distinct destination ports contacted," allowing the model to "see" sequence-dependent attacks. * XGBoost + SHAP: Uses gradient boosting for classification and SHAP (SHapley Additive exPlanations) to provide per-prediction explanations, showing which features pushed the model toward a specific conclusion. * Digital Chain of Custody: Implements a Level 2 hash-chain (SHA-256) that links every step—from ingestion to the final report—ensuring the evidence hasn't been tampered with.

Datasets

The study utilizes three primary benchmark datasets for training and validation: CICIDS2017, UNSW-NB15, and Kitsune. To test the system's "Formal Mode," the authors used real-world evidence including Suricata alerts, CEF firewall logs, and raw packet captures (PCAP) generated from their own infrastructure.

Key Insights and Ideas

The most compelling parts of the paper are the "design decisions" triggered by empirical failures:

  • The Timestamp Leak: The authors found that an initial model achieved a near-perfect 0.984 F1-score on the Kitsune dataset. SHAP analysis revealed the model wasn't learning traffic patterns; it was learning the absolute timestamps of the capture sessions—a forensically useless artifact.
  • The "Honest" Performance Drop: When the model was restricted to the 5 features actually available in real logs (vs. the 78 in the lab), performance dropped, but the model became capable of correctly identifying real SSH brute-force attacks that the "full" model missed.
  • The Silent Binary Bug: During testing against a real packet capture, the authors discovered a bug where the pipeline was silently training a binary classifier (Malicious vs. Normal) even when multiclass labels were available. This was only caught by confronting the model with real-world data, as it was invisible in standard benchmark tests.

Key Results

  • Temporal Aggregation: Adding the temporal aggregation layer caused the multiclass F1-macro on the Kitsune dataset to jump from 0.394 to 0.932, proving that context is king for certain attack types.
  • Model Performance: The default certified model (v1) achieved an F1-macro of 0.9538 on 4 classes. A more complex 18-category model (v3) reached 0.6214, a figure the authors argue is a more "honest" reflection of the difficult multiclass task in a forensic setting.
  • Port-Scan Detection: Specialized binary detectors for port scanning reached an impressive 0.9997 F1-macro.

Limitations and Future Work

  • Chain of Custody: The current system uses Level 2 (hash-chaining). The authors aim to move to Level 3 by incorporating digital signatures for non-repudiation.
  • Legal Scrutiny: While designed for regulatory alignment (like the EU AI Act), MODEXRE has not yet been evaluated by legal practitioners in an actual judicial process.
  • Attack Diversity: The empirical validation focused heavily on port scanning and reconnaissance; other categories like shellcode or backdoors require further depth of analysis.

Summary updated

Abstract

Deploying an explainable intrusion detection model from a validated laboratory setting into a forensic workflow requires guarantees that benchmarks do not test: that the certified model is the exact artifact applied to a case, that heterogeneous evidence can be classified reliably, and that every step is verifiable. We present MODEXRE, a forensic pipeline that extends a previously validated synthetic-data-trained XGBoost classifier with SHAP explainability (arXiv:2607.00763) into a dual-mode architecture: a Laboratory mode, where models are trained and certified on public and synthetic data, and a Formal mode, where a frozen, hash-verified model performs inference on case evidence under a SHA-256 hash-chained chain of custody. Suricata alerts, CEF firewall logs and packet captures are normalized through OCSF. Four design decisions, each triggered by an observed failure, shaped the architecture: a timestamp leak that inflated Kitsune performance (F1-macro 0.984, 0.662 after correction); a feature-coverage gap that made the 78-feature reference model misclassify a real SSH alert; causal temporal flow aggregation, which raised Kitsune multiclass F1-macro from 0.394 to 0.932; and a certification defect, exposed only by a real packet capture, whereby the pipeline had always trained a binary classifier. After correction, the same capture went from 100% of events flagged as malicious to 87% classified as Normal, with the scanning host identified as Reconnaissance. The default four-class certified model reaches F1-macro = 0.9538 on CICIDS2017-derived evidence; the recertified 18-category model reaches 0.6214, reflecting the real multiclass task. Chain-of-custody verification, model integrity hashing and per-prediction SHAP explanations are reported as measured outcomes.
BibTeX
@misc{alonso2026network,
  author = {Jose Luis Vela Alonso and Carmen Pellicer},
  title = {{Network Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of Custody}},
  year = {2026},
  eprint = {2611.06173},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06173},
}
Figure