TL;DR
This paper introduces a unified framework for intrusion detection in both network traffic (NIDS) and system call logs (HIDS) using Continuous Time Bayesian Networks (CTBNs). By modeling normal behavior and flagging deviations, their approach effectively handles the bursty and time-varying nature of system events without requiring fixed time intervals, outperforming existing methods in several scenarios.
What Problem Does This Paper Address?
Intrusion detection systems (IDSs) are crucial for cybersecurity, but traditional methods face challenges. Manually creating rules for every new attack is infeasible, and anomaly detection, which identifies deviations from normal behavior, is more promising. However, modeling the dynamic and often "bursty" nature of computer system activities (like network traffic spikes or rapid sequences of system calls) is difficult with discrete-time models that require fixed intervals. This can lead to missed events during inactive periods or inefficient processing during busy times. The authors aim to address this by developing a flexible, continuous-time modeling approach applicable to both network and host-based intrusion detection.
Methods
The core of the paper is the application of Continuous Time Bayesian Networks (CTBNs) for anomaly detection. CTBNs are well-suited for modeling systems with continuous-time dynamics and dependencies between variables.
- Unified Framework: The authors propose a general CTBN-based anomaly detection technique that can be applied to both NIDS and HIDS.
- Anomaly Detection: Generative models are built from normal training data. Deviations from this learned norm are flagged as anomalies based on their low likelihood under the model.
- NIDS Approach: For network traffic, they construct a hierarchical CTBN model. To handle the complexity and learn parameters from continuous-time evidence, they employ Rao-Blackwellized Particle Filtering (RBPF). This technique efficiently samples a portion of the variables and analytically integrates out the rest, reducing computational burden.
- HIDS Approach: For system call logs, they develop a novel learning method to address the challenge of finite resolution clocks. System calls occurring within the same clock tick are assigned the same timestamp, leading to partially observed timing information. Their method allows learning from this imprecise timing without resorting to time discretization.
- Parameter Learning: For both NIDS and HIDS, they utilize the Expectation-Maximization (EM) algorithm for parameter estimation, leveraging their CTBN models and inference techniques.
Datasets
- NIDS:
- MAWI dataset: A publicly available backbone traffic repository.
- LBNL dataset: Internal enterprise traffic from LBNL.
- HIDS:
- DARPA 1998 BSM dataset: Labeled network-based attack data from MIT Lincoln Laboratory, specifically using Solaris Basic Security Module (BSM) audit logs for system call data.
Key Insights and Ideas
- Continuous Time is Key: The authors strongly advocate for continuous-time models (CTBNs) over discrete-time models for capturing the true dynamics of system events, especially bursty ones. This avoids the inefficiencies and potential blind spots of fixed time slicing.
- Unified Modeling Power: Demonstrating that CTBNs can effectively model vastly different data types (network packets vs. system calls) highlights the generality and power of the framework.
- Handling Imprecise Timing: The novel method for dealing with finite clock resolution in system call logs is a significant contribution, allowing for more accurate modeling of host-based activities without losing temporal context.
- Hierarchical Modeling for NIDS: The hierarchical CTBN for network traffic, which models individual ports and then links them, allows for capturing more complex structural dependencies than previous port-independent models.
- RBPF for Scalability: The use of Rao-Blackwellized Particle Filtering is crucial for making their complex CTBN models computationally tractable for real-world datasets.
Key Results
- NIDS (Worm Detection): Their CTBN approach, particularly with RBPF, generally outperformed or matched other methods (Connection Counting, Nearest Neighbor, Parzen Window, SVM-Spectrum) in detecting injected worm attacks (IP Scanning, Mydoom, Slammer) across MAWI and LBNL datasets, especially when attacks were stealthier. ROC curves showed competitive or superior performance.
- NIDS (Host Identification): The CTBN model demonstrated strong performance in identifying individual hosts based on their network traffic patterns on the LBNL dataset, significantly outperforming SVM spectrum kernel methods.
- HIDS (System Call Anomaly Detection): The CTBN method achieved competitive results on the DARPA BSM dataset, outperforming Nearest Neighbor and SVM with spectrum kernel, and performing comparably to sequence-based methods like "stide" while offering greater flexibility for incorporating prior knowledge.
Limitations and Future Work
- Data Collection Difficulty: The authors note that collecting combined network and system call data for a unified detection system is challenging, but suggest it would be a natural and promising next step.
- Computational Complexity: While RBPF and other approximate inference methods are used, exact inference in CTBNs can still be computationally prohibitive for very large state spaces.
- Model Structure Assumption: The CTBN structure is assumed to be known. Learning the structure of CTBNs is a more complex problem.
- Future Work: The paper explicitly suggests combining NIDS and HIDS data for more robust detection, leveraging the strengths of both approaches within their unified CTBN framework.
Summary updated