TL;DR
This paper tackles the challenge of anomaly detection in aggregated system logs, where individual events are summarized into counts over time. The authors propose a novel "Multi-Graphs Autoencoder" (MGAE) that leverages graph representations of this aggregated data to capture complex relationships between events, outperforming traditional single-graph autoencoders. This approach offers a more robust way to identify unusual patterns in large-scale, aggregated datasets.
What Problem Does This Paper Address?
Modern systems generate massive amounts of data, often in the form of event logs. Analyzing these logs is crucial for debugging, identifying failures, and understanding system behavior. However, the sheer volume of data makes it impractical to analyze each individual event. Consequently, data is often aggregated, typically by counting event occurrences within specific time windows. This aggregation, while improving storage efficiency, presents a significant challenge for anomaly detection. Traditional methods that rely on event sequences or individual event characteristics are no longer directly applicable. The paper addresses this gap by proposing a method to detect anomalies in such aggregated, count-based data by representing it as a graph and using a specialized autoencoder.
Methods
The core of the paper's contribution lies in its novel approach to representing aggregated data as graphs and then using a Multi-Graphs Autoencoder (MGAE) for anomaly detection.
- Graph Representations: The authors develop three distinct graph representations for the aggregated data, each capturing different aspects of event relationships within sessions:
- Relations Graph: Represents sessions as directed graphs where edges capture the ratio of event counts.
- Appearances Graph: Edges represent the co-occurrence frequency of events across sessions.
- Count Graph: Focuses on event counts, with variations in how counts are represented (diagonal vs. full row).
- Multi-Graphs Autoencoder (MGAE): This is the proposed anomaly detection model. It's an unsupervised learning technique that uses Graph Convolutional Networks (GCNs) for both its encoder and decoder. The MGAE takes multiple graph representations of the aggregated data as input and aims to reconstruct them. By learning to reconstruct these multi-graph representations, the model can identify deviations from normal patterns, which are indicative of anomalies.
- Anomaly Detection Model: While the primary focus is on the MGAE's reconstruction capabilities, the paper also introduces a simpler anomaly detection model based on identifying "stable couples" (events that are consistently correlated) and using prediction error checks.
Datasets
The research utilizes a dataset of SQL command logs provided by IBM. This dataset contains information about SQL commands issued by users to a database, aggregated by frequency (count) within specific time ranges, users, and sessions. The paper does not mention the introduction of new benchmarks but focuses on analyzing this specific, real-world dataset.
Key Insights and Ideas
- Graph Representation is Key for Aggregated Data: The paper demonstrates that transforming aggregated, count-based log data into graph structures is a powerful way to capture the complex interdependencies between events that are lost in simple aggregation.
- Multi-Graph Approach Captures Richer Relationships: Using multiple graph representations (Relations, Appearances, Count) as input to the autoencoder allows the MGAE to learn a more comprehensive understanding of the data, leading to better reconstruction and, by extension, anomaly detection.
- MGAE Outperforms Single-Graph Models: The experimental results consistently show that the MGAE, by leveraging multiple graph views, achieves lower reconstruction errors compared to traditional single-graph autoencoders, especially for more complex relationships.
- Correlation Stability is a Strong Indicator: The research highlights that stable correlations between event counts are a significant indicator of normal behavior and can be a foundation for anomaly detection.
Key Results
- The proposed MGAE model achieved a 60% decrease in reconstruction error compared to standard graph autoencoders.
- The MGAE demonstrated superior performance in reconstructing graphs, particularly for sessions with higher degrees of relationships between events.
- Analysis of correlations showed that over 70% of identified correlations remained stable during testing, suggesting their utility for identifying normal or abnormal behavior.
- The "Multi-graphs - Count diagonal" representation consistently yielded the best results across various normalization techniques.
Limitations and Future Work
- Limited Anomaly Detection Testing: The paper acknowledges that the latest model presented was not fully tested as an anomaly detection model, suggesting this as a future direction.
- Difficulty in Full Reconstruction: The authors found it "nearly impossible to fully reconstruct the relations graph" for sessions with a large number of constructs using the proposed methods, especially when MSE error was low. This suggests challenges in handling very complex graphs.
- Threshold Sensitivity: The precision and recall scores were found to be "easily biased by the different thresholds settings."
- Future Work Suggestions:
- Develop a dedicated anomaly detection model based on the MGAE, potentially using reconstruction error as a metric and incorporating baseline anomaly detection algorithms.
- Incorporate feature matrices into graph representations to include additional event/session features.
- Explore existing graph embedding models to handle larger graphs with more nodes and edges.
- Extend the MGAE model to other datasets, including public ones, to further validate its benefits.
- Investigate more complex autoencoder architectures to better capture higher-degree relationships and reduce reconstruction error.
Summary updated