TL;DR
This paper tackles the challenge of detecting anomalies in rapidly growing campus network traffic by leveraging the Map-Reduce model. By applying this distributed computing framework to analyze non-sampled traffic data, they demonstrate an efficient method for identifying malicious or unusual network behavior.
What Problem Does This Paper Address?
The exponential growth of network traffic, driven by advancements in technology and the Internet of Things, presents a significant challenge for traditional network anomaly detection systems. These systems often rely on sampling traffic records, which can lead to detrimental traffic slipping through undetected. Furthermore, campus networks face unique security threats from both external hackers and internal malicious activities, necessitating robust anomaly detection to prevent breakdowns and ensure service availability. The authors aim to address the scalability and efficiency issues in detecting these anomalies within large-scale campus network traffic.
Methods
The core of this paper's approach is the application of the Map-Reduce programming model. This model, originally developed by Google, is designed for processing large datasets in parallel across a cluster of computers. The authors implement a campus network anomaly detection system using Hadoop, an open-source Java-based implementation of Map-Reduce. The system utilizes a DDoS anomaly detection approach that runs on a Map-Reduce cluster. The process involves:
- Map Phase: Input data (network traffic records) is processed by map functions, which generate intermediate key-value pairs.
- Sort Phase: Intermediate key-value pairs are sorted based on their keys.
- Reduce Phase: Reduce functions process the grouped intermediate values to produce the final output, which in this context, is the identification of anomalies.
This approach is novel in its application of Map-Reduce for real-time, non-sampled campus network traffic analysis, building upon previous work that used Hadoop for traffic measurement and DDoS detection.
Datasets
The authors evaluated their system using the CTU-13 dataset. This is a large-scale public dataset captured at CTU University, which contains campus network traffic. The paper notes that this dataset was chosen because captured campus network traffic can contain private information, and using a public dataset allows for reproducible research. The CTU-13 dataset is used to detect campus network traffic anomalies.
Key Insights and Ideas
- Map-Reduce for Scalability: The Map-Reduce model is well-suited to handle the massive scale of modern network traffic, offering a scalable solution where traditional methods might falter.
- Non-Sampled Analysis is Crucial: By processing non-sampled traffic, the system can detect anomalies that might be missed by sampling-based approaches, leading to more comprehensive security.
- Hadoop as a Practical Implementation: Utilizing Hadoop provides a robust and widely adopted framework for implementing the Map-Reduce model, making the proposed system practical and accessible.
- DDoS Detection as a Proxy for Anomalies: The paper leverages DDoS anomaly detection techniques as a foundation, suggesting that these methods can be generalized to identify a broader range of campus network anomalies.
Key Results
The experimental results on the CTU-13 dataset confirm that the Map-Reduce model can efficiently detect campus network anomalies. While specific quantitative metrics like detection accuracy or false positive rates are not detailed in the provided text, Figure 2 shows the progress of a Map-Reduce job processing the CTU-13 dataset, indicating the map and reduce phases progressing through the data. The paper concludes that their proposed system, based on Map-Reduce and Hadoop, is effective for this task.
Limitations and Future Work
Not discussed in the paper.
Summary updated