Untargeted white-box adversarial attack with heuristic defence methods in real-time deep learning based network intrusion detection system

Khushnaseeb Roshan, Aasim Zafar, Shiekh Burhan Ul HaqueComputer Communications 2023 · DOI · arXiv

TL;DR

This paper explores the vulnerability of Deep Learning-based Network Intrusion Detection Systems (NIDS) to adversarial attacks, demonstrating how simple perturbations can significantly degrade their performance. The authors then propose and evaluate three heuristic defense strategies—Adversarial Training, Gaussian Data Augmentation, and High Confidence—to enhance NIDS robustness against these attacks.

What Problem Does This Paper Address?

Network Intrusion Detection Systems (NIDS) are crucial for cybersecurity, but their reliance on Machine Learning (ML) and Deep Learning (DL) models introduces a new attack surface: adversarial attacks. These attacks involve subtly manipulating input data to fool ML models into making incorrect predictions. While adversarial attacks have been extensively studied in computer vision, their application and defense in the context of NIDS, which deal with complex network traffic data, remain a significant research challenge. This paper specifically addresses the problem of defending DL-based NIDS against untargeted white-box adversarial attacks, aiming to understand their vulnerabilities and propose effective defense mechanisms.

Methods

The research employs a multi-faceted approach:

  • Adversarial Attack Techniques: Four powerful white-box adversarial attack methods are implemented:
  • Fast Gradient Sign Method (FGSM): A rapid method that uses the gradient of the loss function to generate adversarial examples with minimal perturbation.
  • Jacobian Saliency Map Attack (JSMA): This method iteratively identifies the most significant features that cause large variations in model predictions.
  • Projected Gradient Descent (PGD): An extension of the Basic Iterative Method, PGD is a robust attack that iteratively searches for perturbations to fool the model.
  • Carlini & Wagner (C&W): A powerful targeted attack that uses L-norm constraints to make perturbations imperceptible.
  • Defense Strategies: Three heuristic defense methods are proposed and evaluated:
  • Adversarial Training (AT): The model is retrained with adversarial samples injected during the training phase to improve robustness.
  • Gaussian Data Augmentation (GDA): Random Gaussian noise is added to input data during inference to smooth model confidence and explore multiple directions of perturbation.
  • High Confidence (HC): This post-processing approach only accepts predictions when the model is highly confident in its output.
  • Real-time Implementation: The study conceptually demonstrates the real-time execution of both white-box and black-box adversarial attacks on network traffic, illustrating packet flow from source to destination.

Datasets

  • CICIDS-2017 Dataset: This dataset, generated by the Canadian Institute for Cybersecurity, is used for experimentation. It is described as a comprehensive and up-to-date dataset containing various types of attacks, making it suitable for evaluating NIDS performance. The dataset is preprocessed, normalized, and split into training, validation, and testing sets.

Key Insights and Ideas

  • NIDS Vulnerability: Even sophisticated DL-based NIDS are susceptible to adversarial attacks, with performance metrics like accuracy, precision, and recall significantly degrading when subjected to these perturbations.
  • Attack Effectiveness: The C&W attack is identified as a powerful adversarial technique, though its impact on NIDS classification reports is comparable to FGSM, JSMA, and PGD.
  • Defense Efficacy: Heuristic defense strategies can significantly improve NIDS robustness. Adversarial Training and High Confidence defenses show similar effectiveness for FGSM, JSMA, and PGD attacks. Gaussian Data Augmentation shows a slight edge over other methods in defending against the C&W attack.
  • Transferability Concept: The paper highlights the concept of adversarial transferability, where examples crafted to fool one model can deceive others, suggesting that white-box attacks can be relevant even in black-box scenarios.
  • Real-time Implications: The study emphasizes the need for robust defense mechanisms in real-time streaming applications, as DL-based NIDS alone may not be sufficient for mission-critical security.

Key Results

  • Pre-attack: The NIDS model achieved high performance on clean data, with weighted accuracy, precision, recall, and F-score around 98.54%.
  • Post-attack: Under adversarial attacks, performance degraded significantly:
  • FGSM: Weighted accuracy dropped to 57.59%.
  • JSMA: Weighted accuracy was 66.58%.
  • PGD: Weighted accuracy was 56.81%.
  • C&W: Weighted accuracy was 61.74%.
  • Post-defense: The proposed defense strategies substantially improved NIDS performance:
  • Adversarial Training: Achieved weighted accuracy of around 98.7% for FGSM, JSMA, and PGD, and 71.56% for C&W.
  • Gaussian Data Augmentation: Achieved weighted accuracy of around 95.71% for FGSM, 97.3% for JSMA, 95.82% for PGD, and 74.11% for C&W.
  • High Confidence: Achieved weighted accuracy of around 98.7% for FGSM, JSMA, and PGD, and 71.52% for C&W.

Limitations and Future Work

  • Adversarial Landscape Complexity: The study focused on four specific attack techniques, while the adversarial landscape is vast and evolving.
  • Static Dataset: The use of static datasets may not fully capture the dynamic nature of real-world network traffic and adversarial behaviors.
  • Simplified Attack Models: The assumption of white-box knowledge might not always hold in practice, where attackers may operate under more restrictive black-box conditions.
  • Defense Evaluation: The study explored various defense strategies but did not conduct an exhaustive analysis of all possible mechanisms.
  • Evaluation Metrics: While standard metrics were used, additional metrics and real-world testing could provide a more comprehensive assessment.

Future Work: * Investigate emerging attack techniques and develop robust adversarial-proof frameworks. * Conduct in-depth analysis of black-box attacks and develop countermeasures for scenarios with limited attacker knowledge. * Integrate explainability and model interpretability into NIDS. * Develop adaptive systems for concept drift in network traffic and attack methodologies. * Explore transfer learning for enhanced NIDS robustness. * Validate NIDS defenses through real-world deployments. * Investigate hybrid defense strategies combining heuristic and certified methods for comprehensive threat coverage.

Summary updated

BibTeX
@inproceedings{roshan2023untargeted,
  author = {Khushnaseeb Roshan and Aasim Zafar and Shiekh Burhan Ul Haque},
  title = {{Untargeted white-box adversarial attack with heuristic defence methods in real-time deep learning based network intrusion detection system}},
  booktitle = {Computer Communications},
  year = {2023},
  doi = {10.1016/j.comcom.2023.09.030},
}
Figure