<title>Protecting against cyber threats in networked information systems</title>

Levent Ertöz, Aleksandar Lazarević, Eric Eilertson, Pang‐Ning Tan, Paul Dokas, Vipin Kumar, Jaideep SrivastavaProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2003 · DOI

TL;DR

This paper introduces the MINDS (Minnesota Intrusion Detection System), a novel approach to detecting cyber threats in networked systems using data mining. Unlike traditional signature-based methods that miss new attacks, MINDS leverages anomaly detection and rare class learning to identify previously unknown intrusions in live network traffic, demonstrating significant promise in enhancing cybersecurity.

What Problem Does This Paper Address?

The increasing reliance on networked information systems has made them prime targets for cyber attacks. Traditional intrusion detection systems (IDS) rely on signatures of known threats, rendering them ineffective against novel and emerging attacks. This creates a critical vulnerability, as attackers can exploit these unknown threats with a significant latency before defenses can be updated. The sheer volume and complexity of network traffic also make manual analysis impractical, necessitating automated solutions.

Methods

The MINDS system employs a three-pronged data mining approach:

  • Known Attack Detection Module (Learning from Rare Class): This module focuses on building predictive models for rare events, specifically known cyber attacks that constitute a small fraction of network traffic. It utilizes novel classification algorithms designed for imbalanced datasets, going beyond traditional methods that struggle with such distributions.
  • Anomaly Detection Algorithms: This core component builds models of normal network behavior and flags any significant deviations as potential intrusions. It uses outlier detection schemes that analyze the full dimensionality of data points and local neighborhood densities to identify anomalies.
  • Association Pattern Analysis for Summarization of Attacks: After identifying anomalous connections, this module uses association rules to summarize and characterize these anomalies. This helps in understanding the nature of emerging threats and can be used to generate new signatures for future detection.

The system preprocesses raw network traffic data, extracts features, and then feeds it into these modules.

Datasets

The paper primarily discusses experiments conducted on live network traffic at the University of Minnesota. While specific details about the dataset size or composition are not provided, it's highlighted that this data is characterized by high volume, dimensionality, and heterogeneity, representing millions of network connections daily. The authors also mention that the data is labeled as "normal" or "intrusion" for training purposes, but the challenge of obtaining high-quality, labeled data is acknowledged.

Key Insights and Ideas

  • The limitations of signature-based detection are severe: Traditional IDS are fundamentally blind to novel threats, creating a critical window of vulnerability.
  • Data mining offers a powerful alternative: By learning patterns of normal behavior and identifying deviations, data mining can detect previously unseen attacks.
  • Handling imbalanced data is crucial: Cyber attacks are rare events, making it challenging for standard data mining techniques. MINDS addresses this with specialized rare class learning algorithms.
  • Anomaly detection is not perfect but highly valuable: While anomaly detection can generate false positives, it's essential for identifying novel threats that signature-based systems miss.
  • Summarizing anomalies is key to actionable intelligence: Simply flagging anomalies isn't enough; understanding their patterns helps in creating new defenses and understanding attacker strategies.

Key Results

Experimental results on live network traffic at the University of Minnesota show that MINDS is highly promising in detecting novel intrusions. The paper highlights several instances where MINDS successfully identified attacks that were missed by SNORT, a state-of-the-art signature-based IDS.

  • MINDS detected slow-rate scanning attacks (e.g., Microsoft-DS service on port 445/TCP) that SNORT's scan detector missed.
  • It identified variations of known worms using non-standard ports, which SNORT, lacking specific signatures, could not detect.
  • MINDS demonstrated superior performance in detecting policy violations (e.g., rogue VPN or FTP servers on non-standard ports) by identifying unusual network behavior, a task that would require specific rules for each activity in SNORT.
  • The association pattern analysis successfully characterized anomalous connections, aiding security analysts in understanding emerging threats and creating new attack signatures.

Limitations and Future Work

  • No quantitative detection rate: The paper acknowledges that it's not possible to provide an exact detection rate for MINDS due to the nature of anomaly detection and the difficulty in estimating the fraction of attacks identified as anomalous.
  • Overlap with SNORT: While MINDS can detect what SNORT misses, there is a small overlap with SNORT's capabilities, suggesting that a combined approach could enhance overall coverage.
  • Future Work: The authors plan to continue developing their framework for defending against cyber threats, focusing on high-performance computing for on-line and distributed analysis. They are also developing a visualization tool to assist analysts in better understanding detected anomalous behavior.

Summary updated

Abstract

This paper provides an overview of our efforts in detecting cyber attacks in networked information systems. Traditional signature based techniques for detecting cyber attacks can only detect previously known intrusions and are useless against novel attacks and emerging threats. Our current research at the University of Minnesota is focused on developing data mining techniques to automatically detect attacks against computer networks and systems. This research is being conducted as a part of MINDS (Minnesota Intrusion Detection System) project at the University of Minnesota. Experimental results on live network traffic at the University of Minnesota show that the new techniques show great promise in detecting novel intrusions. In particular, during the past few months our techniques have been successful in automatically identifying several novel intrusions that could not be detected using state-of-the-art tools such as SNORT.
BibTeX
@inproceedings{ertoz2003lt,
  author = {Levent Ertöz and Aleksandar Lazarević and Eric Eilertson and Pang‐Ning Tan and Paul Dokas and Vipin Kumar and Jaideep Srivastava},
  title = {{\<title\>Protecting against cyber threats in networked information systems\</title\>}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2003},
  doi = {10.1117/12.487410},
}
Figure