An Abnormal Network Traffic Detection Algorithm Based on Big Data Analysis

Haipeng Yao, Yiqing Liu, Chao FangInt. J. Comput. Commun. Control 2016 · DOI

Abstract

Anomaly network detection is a very important way to analyze and detect malicious behavior in network. How to effectively detect anomaly network flow under the pressure of big data is a very important area, which has attracted more and more researchers’ attention. In this paper, we propose a new model based on big data analysis, which can avoid the influence brought by adjustment of network traffic distribution, increase detection accuracy and reduce the false negative rate. Simulation results reveal that, compared with k-means, decision tree and random forest algorithms, the proposed model has a much better performance, which can achieve a detection rate of 95.4% on normal data, 98.6% on DoS attack, 93.9% on Probe attack, 56.1% on U2R attack, and 77.2% on R2L attack.

Introduction

As network traffic enters the "Big Data" era, traditional intrusion detection systems (IDS) face significant hurdles. The primary challenges include extreme data imbalance—where normal traffic vastly outweighs attack data—and shifting distributions, where the statistical nature of traffic changes rapidly, rendering static models obsolete. Traditional methods typically fall into misuse detection (high accuracy for known signatures) or anomaly detection (capable of finding new attacks but prone to high false-positive rates). This paper introduces a "Mixed Compensation Model" designed to bridge this gap, maintaining robustness even as traffic distributions shift.

Methodology: The Mixed Compensation Model

The proposed approach utilizes a three-stage model combining K-means clustering, Decision Trees (C4.5), and Random Forests to decouple initial detection from specific classification.

  1. Normal Traffic Selection Model: Acts as a binary filter using K-means and Random Forest to distinguish normal from abnormal behavior. By increasing the number of clusters ($K$), a stricter definition of "normal" is established.
  2. Abnormal Traffic Selection Model: Trained exclusively on abnormal data, this stage focuses on classifying specific attack types (e.g., DoS, Probe). By removing normal data from this training phase, the model can better identify features of rare attack classes.
  3. Mixed Compensation Model: A fusion layer that applies logical rules to combine outputs. It prioritizes reducing false negatives, ensuring that potential attacks are not missed when models disagree.

Dataset and Feature Analysis

The model was evaluated using the KDDCUP99 dataset, which includes 41 features across basic, content, and time-based categories. The training set utilized the "10 percent" subset, while the test set contained over 311,000 records.

Distribution of the KDDCUP99 dataset showing significant class imbalance.

As shown in the table above, the dataset is heavily skewed toward DoS attacks and normal traffic, with rare attacks like U2R and R2L representing a tiny fraction of the total records. This imbalance is a primary motivator for the decoupled selection models.

A key insight from the research is that feature importance varies significantly depending on the detection task. Using the Gini index, the authors demonstrated that the variables most critical for distinguishing normal from abnormal traffic differ from those needed to classify specific attack types.

Comparison of feature importance across different selection models.

Results and Discussion

The Mixed Compensation Model demonstrated superior performance, particularly in detecting rare attack categories that traditional single-algorithm approaches often miss.

Category Proposed Model Accuracy KDDCUP99 Winner
Normal 95.4% 99.5%
DoS 98.6% 97.1%
Probe 93.9% 83.3%
U2R 56.1% 13.2%
R2L 77.2% 8.4%

While the model was slightly less accurate on normal data than the competition winner, its ability to detect U2R and R2L attacks—often the most dangerous and difficult to catch—represents a significant advancement. The hybridization of filtering and classification effectively mitigates the "drowning out" effect of high-volume normal traffic.

Conclusion

The Mixed Compensation Model offers a robust framework for intrusion detection in big data environments. By decoupling the identification of abnormal traffic from its classification, the system achieves higher sensitivity to rare attacks. Future work should address the limitations of the aging KDDCUP99 dataset by testing against modern, encrypted network traffic and further improving detection rates for U2R attacks.

Summary updated

BibTeX
@article{yao2016abnormal,
  author = {Haipeng Yao and Yiqing Liu and Chao Fang},
  title = {{An Abnormal Network Traffic Detection Algorithm Based on Big Data Analysis}},
  journal = {Int. J. Comput. Commun. Control},
  year = {2016},
  doi = {10.15837/ijccc.2016.4.2315},
}
Figure