Introduction
Modern network security faces a dual challenge: the overwhelming volume of traffic and the extreme rarity of high-stakes attacks. Traditional Machine Learning (ML) models, such as Artificial Neural Networks (ANNs), often struggle with "data noise"—redundant or irrelevant audit log entries—and the "Rare Attack" problem, where critical threats like User-to-Root (U2R) unauthorized access are buried under a mountain of normal traffic or common Denial-of-Service (DoS) events. This paper proposes a novel Query-Based Sampling (QBS) filter that applies signal processing principles to curate high-quality training data, focusing on the most informative samples to improve detection accuracy and computational efficiency.
The QBS Filter Methodology
The central innovation of this research is the QBS filter, which treats the selection of training data as a signal processing task. By drawing an analogy to Pulse Code Modulation (PCM) in signal quantization, the filter identifies "distortion" or noise within the training set.
The system operates as an active learner. Rather than processing a static dataset, it uses a partially trained classifier to identify ambiguous samples—those residing near the decision boundary between "normal" and "attack" classes. It then queries an "oracle" (a ground-truth source) to label these high-value points.
A key component of this strategy is the identification of conjugate data pairs. These are pairs of data points that are spatially close in the feature space but belong to different classes. By focusing the training on these boundary-defining samples, the model can establish a sharper, more accurate distinction between benign and malicious activity.
Experimental Evaluation
The researchers validated the QBS filter using the KDD Cup 1999 dataset, a standard benchmark for intrusion detection. The dataset is notoriously imbalanced; for example, U2R attacks constitute a mere 0.011% of the training data. This imbalance typically causes standard algorithms to ignore rare classes in favor of maximizing overall accuracy on the majority classes.
Results and Discussion
The QBS method was compared against Stratified Random Sampling (StRS), a common baseline. While both methods achieved high overall accuracy (QBS at 91.55% vs. StRS at 91.40%), the QBS filter demonstrated a significant advantage in detecting rare attacks.
For the critical U2R category, QBS achieved a True Positive (TP) rate of 0.399, a substantial improvement over the 0.263 rate of StRS. Furthermore, QBS exhibited higher precision for normal traffic during training (0.996 vs. 0.974), indicating a lower rate of false alarms. By selecting a concise, "qualified" subset of data, the method not only improves detection of subtle threats but also reduces the computational burden, making real-time model updates more feasible.
Conclusion
This research demonstrates that in network intrusion detection, quality of data often outweighs quantity. By leveraging signal processing concepts like quantization to filter out noise and focus on decision boundaries, the QBS filter enables neural networks to better generalize from imbalanced datasets. Future work will focus on refining the "oracle" mechanism and better distinguishing between erratic but benign user behavior and true malicious anomalies.
Summary updated