TL;DR
This paper introduces an optimized correlation engine (OC) for Security Information and Event Management (SIEM) systems, designed to significantly improve the detection of multi-layered cyberattacks. By replacing traditional regex matching with the high-performance Hyperscan library for parallel log data scanning, the OC achieves a substantial speedup in real-time response and execution time, making SIEM systems more effective and efficient.
What Problem Does This Paper Address?
The increasing complexity of modern networks and the sophistication of cyberattacks pose a significant challenge for traditional security systems. While SIEM systems are crucial for aggregating and analyzing security logs, their correlation engines, often relying on sequential regex matching, can become performance bottlenecks. This slowness hinders timely detection and response to evolving threats, especially multi-layered attacks that span different network layers. The paper aims to address this performance gap by developing a more efficient correlation engine that can keep pace with the volume and speed of incoming log data.
Methods
The core innovation of this paper is the development of an "Optimized Correlator" (OC). The OC replaces the traditional, sequential regex matching sub-module found in conventional SIEM correlation engines (like SEC, used as a baseline) with the Hyperscan library. Hyperscan is a high-performance, multi-pattern regex matching library that leverages:
- Parallelism: It can scan multiple data blocks concurrently against a read-only rule database.
- Graph Decomposition: It breaks down regular expressions into smaller components, enabling faster matching.
- SIMD Instructions: It utilizes Single Instruction, Multiple Data (SIMD) operations for accelerated string and finite automata matching.
The OC is implemented in C and features a multi-threaded architecture, allowing it to exploit parallelism and process log data more efficiently. The design involves splitting incoming log data into blocks, which are then processed in parallel by multiple threads against the compiled pattern database.
Datasets
The paper does not mention specific public datasets for training or evaluation. Instead, it uses log files of varying sizes (102 MB, 256 MB, 512 MB, and 1024 MB) generated from their experimental setup. These log files contain data simulating failed login attempts and other OS logs, as well as logs generated from simulated multi-layered attacks.
Key Insights and Ideas
- Hyperscan is a Game-Changer for SIEM Correlation: The paper strongly suggests that leveraging high-performance libraries like Hyperscan is a critical step in modernizing SIEM correlation engines. Its ability to perform parallel, multi-pattern matching is a significant leap over traditional sequential methods.
- Performance Bottlenecks are Real and Impactful: The research clearly demonstrates how the efficiency of the correlation engine directly impacts the overall effectiveness of a SIEM system, particularly in real-time threat detection.
- Multi-Layered Attack Detection is Crucial: The paper highlights the importance of detecting attacks that span multiple network layers, a capability that requires a robust and fast correlation engine.
- Intel SIMD Architecture is Key: The reliance of Hyperscan on Intel's SIMD architecture is noted as a specific requirement for the OC's optimal performance, which is also identified as a limitation.
Key Results
The experimental results show a significant performance improvement of the proposed OC over the traditional SEC engine:
- Real-time Response: The OC is approximately 21 times faster than SEC.
- System/Execution Time: The OC is about 2.5 times more efficient than SEC.
The OC successfully detected various multi-layered attacks, including:
- Denial-of-Service (DoS) attacks (Simple GoldenEye, Random GoldenEye, Slow HTTP, Slow Loris)
- FTP login attacks
- Failed login attempts (SSH/login)
- MAC flooding attacks
- STP root take-over attacks
Limitations and Future Work
- Intel Processor Dependency: A key limitation is that the OC's performance relies on Intel processors with SIMD architecture, as Hyperscan leverages these specific instructions. This means the OC may not perform optimally on non-Intel architectures.
- Future Work: The authors suggest extending the OC's capabilities by:
- Testing it against more cyberattacks targeting multiple OSI layers.
- Exploring cloud support for the OC.
- Investigating additional performance metrics.
- Utilizing the OC in other domains requiring extensive file scanning, such as SOAR systems, digital forensics, malware analysis, and data mining.
Summary updated