TL;DR
This paper introduces a novel changepoint detection framework to distinguish between human-initiated and automated periodic events in cybersecurity data. By modeling these events as distinct subsequences, the method effectively identifies user actions that trigger automated network activity, offering a more robust approach to anomaly detection and data analysis.
What Problem Does This Paper Address?
Cybersecurity anomaly detection often relies on building statistical models of normal network behavior and identifying deviations. A significant challenge in this domain is the presence of mixed event types: human-generated actions and automated network processes, often appearing as periodic "polling" behavior. Existing methods struggle to disentangle these, potentially leading to misinterpretations and less effective anomaly detection. This paper tackles the problem of identifying and separating these distinct event types, specifically focusing on how human actions initiate periodic automated subsequences within network event data.
Methods
The core of this paper is a changepoint detection framework designed to identify periodic subsequences within event time series. The authors propose a generative model for event times that can exhibit two types of polling: "fixed phase polling" (where event times are periodic with random error, and errors don't propagate) and "fixed duration polling" (where event times are periodic with random error, and errors do propagate).
Key methodological components include:
- Fourier Analysis: Used as an initial step to detect overall polling behavior.
- Directional Statistics (von Mises Distribution): Employed to model the error terms in periodic events as angular displacements, providing a more robust way to handle noise.
- Changepoint Detection Algorithms: The paper focuses on the Pruned Exact Linear Time (PELT) algorithm for its computational efficiency (linear in data size) and its ability to find global optima. This algorithm partitions the event sequence into segments, each assumed to follow a specific generative model.
- Iterative Inference: The framework involves an iterative process of changepoint detection, Bayesian estimation of nuisance parameters (like probability of event occurrence, periodicity, etc.), and updating the estimated periodicity. This iterative approach accounts for the dependencies between these quantities.
- Robustness to Data Issues: The proposed models are designed to be robust to duplicate and missing data, common issues in real-world network logs.
Datasets
The primary dataset used for evaluation is authentication log data collected over 58 days from the Los Alamos National Laboratory (LANL) internal computer network. This dataset contains information on source and destination usernames/computers and event timestamps. The authors also generated simulated data based on their proposed generative models for fixed phase polling to evaluate the performance of their changepoint detection algorithms.
Key Insights and Ideas
- Human Action as a "Seed" for Automation: The fundamental insight is that user-driven events can be identified as the start of periodic subsequences, which are then followed by automated, periodic network activity. This provides a clear way to distinguish between human and machine behavior.
- Periodic Subsequences as a Key Feature: Recognizing that network traffic isn't just one long periodic signal but rather a series of periodic bursts separated by inactivity is crucial. The framework's ability to detect these discrete subsequences is a major advancement.
- Directional Statistics for Noise Modeling: Using angular displacements and the von Mises distribution to model errors in periodic events is an elegant way to handle the cyclical nature of time-based data and the inherent noise.
- PELT Algorithm's Efficiency: The choice of the PELT algorithm highlights the practical need for computationally efficient methods when dealing with large network datasets, especially when the number of changepoints is unknown and potentially large.
- Robustness is Key: The explicit consideration and demonstration of robustness to duplicate and missing data are critical for applying this method to real-world, often messy, network logs.
Key Results
- Successful Identification of Periodic Subsequences: The PELT algorithm, when applied to both simulated and real LANL data, successfully identified meaningful periodic subsequences.
- Distinguishing Human vs. Automated Events: In the LANL data example, the inferred user-driven events (start of subsequences) showed a diurnal pattern more consistent with human behavior, while the overall event distribution was dominated by automated polling.
- Robustness Demonstrated: The method showed robustness to duplicate data (e.g., between days 5 and 10) and missing data (e.g., between days 35 and 40) in the LANL dataset.
- Computational Efficiency: The PELT algorithm demonstrated competitive or superior run times compared to Binary Segmentation (BS), especially as the number of changepoints increased.
- Parameter Sensitivity: While the algorithm is generally robust, the study showed that variations in prior hyper-parameters had a very small effect on the final inference, indicating stability.
Limitations and Future Work
- Correlation within Beaconing Periods: The authors acknowledge that within a beaconing period, events might not be independent and could arrive in bursts. They suggest a future extension to a hierarchical model that accounts for this correlation by considering clusters of event time variables.
- Unknown Number of Changepoints: While PELT is efficient, the computational cost can still be a factor when dealing with an extremely large number of changepoints, though it's linear.
- No True Labels: For the real data analysis, the absence of true classification labels meant that the validation of inferred user-driven events relied on their consistency with human behavior patterns (diurnal distribution).
- Future Work: The paper suggests incorporating correlation within beaconing periods and exploring more complex hierarchical models for event times. The ultimate goal is to use this separated data for more robust anomaly detection and modeling of user-driven and automated behaviors independently.
Summary updated