A Novel Deep Learning based Model to Defend Network Intrusion Detection System against Adversarial Attacks

Khushnaseeb Roshan, Aasim Zafar, Shiekh Burhan Ul HaquearXiv 2023 · DOI · arXiv

TL;DR

This paper investigates the vulnerability of Deep Learning-based Network Intrusion Detection Systems (NIDS) to adversarial attacks. The authors demonstrate how common attack methods can significantly degrade NIDS performance and propose adversarial training as an effective defense mechanism to improve robustness.

What Problem Does This Paper Address?

Network Intrusion Detection Systems (NIDS) are crucial for cybersecurity, but modern Machine Learning (ML) and Deep Learning (DL) based NIDS are susceptible to adversarial attacks. These attacks involve subtly manipulating network traffic data to trick the NIDS into misclassifying malicious activity as benign, or vice-versa. This vulnerability poses a significant risk to critical infrastructure and sensitive data. The paper aims to understand the impact of various adversarial attack algorithms on DL-based NIDS and evaluate a defense strategy to mitigate these threats.

Methods

The researchers employed four well-known adversarial attack algorithms:

  • Fast Gradient Sign Method (FGSM): A quick method to generate adversarial examples by using the sign of the gradient of the loss function.
  • Jacobian Saliency Map Attack (JSMA): An efficient attack that identifies and perturbs the most influential input features to cause misclassification.
  • Projected Gradient Descent (PGD): An iterative attack that aims to find strong adversarial examples by optimizing the perturbation within a defined constraint.
  • Carlini & Wagner (C&W): A powerful attack known for generating imperceptible adversarial examples, often used to evaluate the robustness of defense mechanisms.

As a defense strategy, the paper implemented Adversarial Training. This involves retraining the NIDS model with adversarially generated samples, effectively teaching the model to be more resilient to such perturbations. The NIDS model itself is a supervised Deep Neural Network (DNN) with an optimal hyperparameter configuration determined through random search.

Datasets

The study utilized a subset of the Canadian Institute for Cybersecurity Intrusion Detection System 2017 (CICIDS-2017) dataset. This dataset is publicly available and contains a variety of network traffic features, including statistical measurements and flow distributions, covering both benign and malicious traffic with fourteen attack classes.

Key Insights and Ideas

  • DL-based NIDS are highly vulnerable: Even sophisticated DL models can be significantly fooled by carefully crafted adversarial examples, leading to a drastic drop in detection accuracy.
  • Adversarial training is a potent defense: Retraining the NIDS model with adversarial samples significantly improves its robustness and performance against these attacks.
  • Attack effectiveness varies: While all tested attacks degraded performance, the specific impact and the effectiveness of the defense varied across different attack algorithms.
  • Constrained domains are not immune: The research reinforces that adversarial attacks are a concern not only in unconstrained domains like image classification but also in constrained environments like network traffic analysis.

Key Results

The study evaluated the NIDS model in three phases: before attack, after attack, and after defense.

  • Before Attack: The NIDS model achieved high performance metrics, with an accuracy of 98.54% and an AUC score of 98.551%.
  • After Attack: Performance dropped significantly across all tested attacks. For instance, accuracy fell to as low as 57.95% with FGSM and 56.81% with PGD. The AUC scores also decreased substantially, with FGSM dropping to 59.232% and PGD to 58.485%.
  • After Defense (Adversarial Training): The adversarial training significantly boosted the model's performance. Accuracy improved to 98.7% for FGSM, 98.47% for JSMA, and 98.68% for PGD. While C&W saw improvements, its precision reached only up to 80.35%, indicating it remains a challenging attack to fully defend against.

Limitations and Future Work

The authors acknowledge that while adversarial training improved robustness, the defense against the C&W attack was not as complete as for other attacks, with precision reaching only around 80.35%. They suggest that future work could focus on further investigating and enhancing the model's robustness specifically against C&W attacks. Additionally, they propose extending the research to explore other ML and DL architectures and utilize more recent intrusion detection datasets to assess the impact of adversarial attacks and defenses in broader contexts. The phenomenon of adversarial transferability, where perturbations designed for one model can affect others, is also highlighted as an interesting area for future investigation.

Summary updated

Abstract

Network Intrusion Detection System (NIDS) is an essential tool in securing cyberspace from a variety of security risks and unknown cyberattacks. A number of solutions have been implemented for Machine Learning (ML), and Deep Learning (DL) based NIDS. However, all these solutions are vulnerable to adversarial attacks, in which the malicious actor tries to evade or fool the model by injecting adversarial perturbed examples into the system. The main aim of this research work is to study powerful adversarial attack algorithms and their defence method on DL-based NIDS. Fast Gradient Sign Method (FGSM), Jacobian Saliency Map Attack (JSMA), Projected Gradient Descent (PGD) and Carlini & Wagner (C&W) are four powerful adversarial attack methods implemented against the NIDS. As a defence method, Adversarial Training is used to increase the robustness of the NIDS model. The results are summarized in three phases, i.e., 1) before the adversarial attack, 2) after the adversarial attack, and 3) after the adversarial defence. The Canadian Institute for Cybersecurity Intrusion Detection System 2017 (CICIDS-2017) dataset is used for evaluation purposes with various performance measurements like f1-score, accuracy etc.
BibTeX
@misc{roshan2023novel,
  author = {Khushnaseeb Roshan and Aasim Zafar and Shiekh Burhan Ul Haque},
  title = {{A Novel Deep Learning based Model to Defend Network Intrusion Detection System against Adversarial Attacks}},
  year = {2023},
  eprint = {2308.00077},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.00077},
}
Figure