A combined feature selection method based on clustering in intrusion detection

Ting Huang, Wenbo Chen, Ruisheng Zhang2017 · DOI

Abstract

The rapid development of information technology generates high dimension and large scale data, which puts severer challenges to network security. Feature selection is proposed for the reduction of data dimension so that features of original data are utmostly retained and improving the effectiveness of data processing. This paper proposes a new method of feature extraction by combining two algorithms. Firstly, removes some noise and irrelevant features after researching for correlation between features and categories; Secondly, furtherly optimize subsets to select key features through feature selection algorithm, whose Evaluate Function is based on clustering algorithm. KDDCUP99 10% datasets is used to testify the experiment, whose result shows the method guarantees effective detection rate and reducing the data dimension effectively at the same time, the effectiveness of data detection is improved.

1. Introduction

In the realm of network security, intrusion detection systems (IDS) are increasingly overwhelmed by high-dimensional and massive-scale data traffic. This "curse of dimensionality" introduces two critical challenges: significant computational latency, which hinders real-time security responses, and reduced detection accuracy due to noise and redundant features. When an IDS attempts to process every available feature of a data packet, it often encounters a "deadly weakness" in real-time processing speed. This paper introduces a hybrid feature selection approach that combines the ReliefF algorithm with K-means clustering to identify a "minimum feature subset" that maintains high detection performance with minimal overhead.

2. Proposed Methodology

The proposed method utilizes a two-stage strategy combining Filter and Wrapper approaches to optimize the feature set, aiming to address both irrelevance and redundancy.

2.1 Stage 1: ReliefF Filter

The ReliefF algorithm serves as the initial filter to remove noise and irrelevant features. It assigns weights to features based on their ability to distinguish between similar and different classes. While ReliefF is excellent at identifying irrelevance, it often fails to detect redundancy—where multiple features provide nearly identical information.

2.2 Stage 2: K-means Wrapper Refinement

To eliminate redundancy, the output of the ReliefF stage is further refined using a heuristic search guided by K-means clustering. A cost function is defined to compare clustering results against actual data labels. By starting with the high-weight features from Stage 1 and iteratively "flipping" bits (adding or removing features) to minimize this cost, the algorithm converges on an optimal, lean subset. This hybrid approach overcomes the inherent instability of standalone clustering by providing a high-quality starting point from the ReliefF stage.

Flowchart of the iterative feature selection and evaluation process.

3. Experimental Setup

The study evaluates the proposed method using the KDDCUP99 (10% subset) benchmark dataset, which contains approximately 490,000 items and 41 original features. Data preprocessing included mapping character-based features to numerical values and performing standardization to ensure accurate distance calculations for K-means. A sampled dataset of 30,000 training and 20,000 testing items (with a 4:1 normal-to-abnormal ratio) was used to simulate realistic network conditions.

4. Results and Discussion

The experimental results demonstrate that the hybrid method significantly reduces dimensionality while improving accuracy compared to standalone methods.

Method Dimensions Modeling Time (s) Accuracy (%)
Original Features 41 4.23 99.60%
ReliefF Only 26 2.89 99.69%
K-means Only 23 2.39 99.69%
Proposed Method 12 1.35 99.77%

4.1 Key Insights

  • Stability through Hybridization: Standalone K-means is often unstable due to random initialization. Initializing the search with ReliefF weights results in a remarkably stable and accurate detection rate.
  • Efficiency of Reduction: By reducing the feature set from 41 to 12 dimensions, the modeling time was cut by nearly 70% (from 4.23s to 1.35s).
  • Information vs. Distraction: The improvement in accuracy from 99.60% to 99.77% suggests that the removed features were primarily "distractions" rather than useful information. The paper demonstrates that a minimal feature set can outperform the original bulky dataset.

5. Conclusion

The combined ReliefF and K-means feature selection method effectively addresses the challenges of high-dimensional data in intrusion detection. It successfully balances dimension reduction with high detection rates. Future work will focus on adapting the cost function for multi-class classification (identifying specific attack types like DoS or R2L) and testing the method against more modern datasets to ensure robustness against contemporary "zero-day" threats.

Summary updated

Abstract

The rapid development of information technology generates high dimension and large scale data, which puts severer challenges to network security. Feature selection is proposed for the reduction of data dimension so that features of original data are utmostly retained and improving the effectiveness of data processing. This paper proposes a new method of feature extraction by combining two algorithms. Firstly, removes some noise and irrelevant features after researching for correlation between features and categories; Secondly, furtherly optimize subsets to select key features through feature selection algorithm, whose Evaluate Function is based on clustering algorithm. KDDCUP9910% datasets is used to testify the experiment, whose result shows the method guarantees effective detection rate and reducing the data dimension effectively at the same time , the effectiveness of data detection is improved.
BibTeX
@misc{huang2017combined,
  author = {Ting Huang and Wenbo Chen and Ruisheng Zhang},
  title = {{A combined feature selection method based on clustering in intrusion detection}},
  year = {2017},
  doi = {10.2991/amcce-17.2017.11},
}
Figure