Abstract
Distributed intrusion detection systems (IDS) are essential for monitoring and responding to anomalies in real-time across large networks. However, the proliferation of IoT devices often leads to an overwhelming volume of alerts and a high rate of false alarms. This paper proposes a framework to mitigate "alert fatigue" by limiting message generation without sacrificing detection accuracy. The system utilizes fog-level nodes for alert collection and analysis, employs a multi-factor prioritization scheme, and leverages Kademlia-based Distributed Hash Tables (DHT) for efficient alert fusion. Experimental results using the DARPA 1999 dataset demonstrate a 62% reduction in message traffic and a false positive accuracy exceeding 80%.
Introduction
As the Internet of Things (IoT) continues to expand, security becomes a paramount concern. Distributed Intrusion Detection Systems (DIDS) are deployed to safeguard these networks, yet they frequently suffer from two critical issues: alert flooding and high false-positive rates. Coordinated attacks can trigger thousands of redundant alerts, overwhelming both network bandwidth and security analysts. Centralized processing of these alerts often creates bottlenecks and single points of failure. This research addresses these challenges by introducing a decentralized framework that processes and fuses alerts at the edge of the network (the "Fog" layer) before they reach the core.
Proposed Framework
The architecture consists of three primary layers: IoT sensors, Fog nodes, and mid-level "Collectors." The framework focuses on reducing redundancy and improving the quality of reported security events through a structured pipeline involving normalization, prioritization, filtering, and fusion.
Multi-Factor Prioritization
To distinguish critical threats from noise, the system calculates a "Cumulative Value" (cv) for each alert based on five key factors:
- Target Importance: The criticality of the asset under attack.
- Event Severity: Standardized scores (e.g., CVE) for the detected vulnerability.
- Detection Confidence: The reliability of the specific sensor's detection algorithm.
- Alert Correlation: Whether the alert fits into a known multi-step attack pattern.
- Service History: The frequency of recent attacks against the targeted service.
Alerts that do not cross a specific threshold are filtered out, while high-priority alerts proceed to the fusion stage. A rescoring mechanism provides feedback to refine the normalization process based on correlation results.
Kademlia-based Alert Fusion
A core innovation is the use of Kademlia-based Distributed Hash Tables (DHTs) for alert routing. By using attack characteristics (such as the destination port) as the DHT key, the network inherently routes related alerts to the same collector node. This decentralized approach ensures that similar alerts are fused into a single, comprehensive report, significantly reducing the volume of data sent to the core network. Kademlia was specifically chosen for its resilience to node churn and lower message overhead compared to other DHT protocols like Chord.
Evaluation and Results
The framework was validated using the DARPA 1999 dataset and real-world DShield logs. The DShield logs, comprising over 600 million entries, helped confirm the stability of port scan patterns and the efficiency of the DHT-based load balancing.
Key findings include:
- Traffic Reduction: The system achieved a 62% reduction in the total number of generated messages.
- Detection Accuracy: False positive accuracy was maintained at over 80%.
- Priority Granularity: Unlike standard tools like Snort, which often provide broad priority categories, the proposed system successfully mapped alerts to a 1-to-10 scale, enabling more precise resource allocation.
- Efficiency: Mathematical modeling confirmed that Kademlia's routing overhead ($n + \log_2 N$) is significantly lower than Chord’s ($n \cdot \log_2 N$) in high-traffic IoT scenarios.
Conclusion
By shifting alert processing and fusion to the fog layer, the proposed framework effectively addresses the scalability issues of modern DIDS. The combination of context-aware prioritization and DHT-based routing provides a robust solution for managing IoT security traffic. Future work will focus on optimizing the computational costs of prioritization, integrating modern IoT-specific datasets, and transitioning to non-structured storage solutions like MongoDB to better handle high-velocity data.
Summary updated