Adaptive Alert Throttling for Intrusion Detection Systems

Gianni Tedesco, Uwe AickelinSsrn 2003 · DOI · OpenReview

TL;DR

Intrusion Detection Systems (IDS) are vulnerable to alert flooding attacks that overwhelm their communication channels, rendering them useless. This paper proposes an adaptive throttling mechanism using token bucket filters to manage alert throughput, significantly improving system resilience and making such attacks prohibitively expensive.

What Problem Does This Paper Address?

As cybersecurity threats evolve, Intrusion Detection Systems (IDS) have become a critical component of network defense. However, these systems themselves are not immune to attack. One particularly insidious threat is the "alert flooding" attack, where an attacker bombards the IDS with a massive volume of false alerts. This deluge of information can overwhelm the IDS's capacity to process and transmit legitimate alerts to human operators or other security systems. Essentially, the communication channel between the IDS and its operator becomes a bottleneck, leading to a denial-of-service (DoS) condition for the IDS itself. This renders the IDS incapable of detecting real threats, effectively blinding the security team. The paper aims to address this vulnerability by developing techniques to enhance the IDS's alert throughput and capacity, making it much harder and more resource-intensive for attackers to successfully execute such an attack.

Methods

The paper introduces two primary techniques to combat alert flooding:

  1. Token Bucket Filter for Adaptive Throttling: This is the core of their proposed solution. A token bucket filter is an algorithm used for rate limiting. In this context, it's applied to the alerts generated by the IDS.
  2. How it works: Tokens are generated at a fixed rate and stored in a "bucket." Each alert requires a token to be processed and sent. If the alert rate exceeds the token rate, alerts are considered "over-limit."
    • Adaptive Nature: The key innovation is making this adaptive. When the IDS is operating normally, it can log all alerts. However, if the token bucket filter detects a high rate of alerts (indicating a potential flood), it can switch to a more "terse" behavior, either dropping over-limit alerts or compressing them.
  3. Configuration: The token bucket filter can be applied at various granularities: per signature, per attack type, or globally. This allows for flexible configuration based on the specific IDS and network environment.
  4. Alert Compression (Run-Length Encoding with Delta Compression): Instead of simply dropping over-limit alerts, the paper proposes compressing them to retain valuable information.
  5. Run-Length Encoding (RLE): If many alerts are identical (as is common in repetitive attack patterns), RLE can represent a sequence of identical alerts with a single symbol and a count. This is similar to how syslog might report "last message repeated N times."
    • Delta Compression: To further enhance compression, especially when alerts are not perfectly identical but share similarities (e.g., modulated source/destination addresses), delta compression is employed. This technique stores the first alert in a run and then encodes the differences (deltas) between subsequent alerts. This is particularly effective for timestamps, allowing for accurate reconstruction of timing information.

Datasets

The paper mentions using a "freely available Shmoo Group 'CCTF' data set which contains a high bandwidth Internet packet fragmentation attack constituting an alert flood." This dataset contained 587,482 packets transmitted over 80 seconds. The experiments were also conducted using a full signature database of around 1,600 signatures.

Key Insights and Ideas

  • The IDS communication channel is a prime target: The paper highlights that the very mechanism designed to report threats can be turned into a weapon against the security system itself.
  • Adaptive throttling is crucial: A static rate limit isn't sufficient. The system needs to dynamically adjust its alert output based on the detected threat level.
  • Dropping alerts is a last resort; compression is better: While dropping alerts can prevent system overload, it means losing valuable forensic data. Compressing alerts, especially using RLE with delta encoding, offers a way to significantly reduce data volume while preserving most, if not all, fidelity.
  • Attackers exploit repetition: The effectiveness of RLE and delta compression relies on the assumption that alert flooding attacks often involve repetitive or similar patterns, which is a reasonable assumption for many automated attack tools.

Key Results

The paper presents experimental results using a token bucket filter configured with a token rate of 1 per second and a bucket size of 10 tokens.

  • Significant Data Reduction: In one test run, the token bucket filter reduced the amount of logged data by "several orders of magnitude."
  • Improved Performance: The run time decreased disproportionately to the CPU time, indicating that the IDS spent less time waiting for I/O completion due to the reduced alert volume. Specifically, the run time was reduced by 60% while CPU time reduced by only 20%.
  • Compression Effectiveness:
    • RLE Only compression reduced data size from 472,192 KB to 171,578 KB (a compression ratio of 2.75:1).
  • RLE with Timestamp Delta compression further reduced data size to 754.857 KB (a compression ratio of 625.5:1 compared to uncompressed). This is a significant improvement, almost an order of magnitude better than existing techniques.

Limitations and Future Work

  • Optimal Token Bucket Parameters: The authors acknowledge that more investigation is needed to determine optimal parameters for the token bucket filter.
  • Hierarchical Filtering: They suggest that an ideal scheme would be hierarchical, offering greater flexibility in configuration.
  • Lossless Full Packet Payload Compression: While delta compression on alert fields is effective, they note that a method for efficiently compressing full packet payloads without loss of fidelity would be beneficial. They suggest that algorithms like rsync or gzip could be explored for this purpose.
  • Real-world System Tuning: The paper suggests that a real-world system could be configured to allow operators to make trade-offs between efficiency and data fidelity based on operational parameters and perceived threat levels.

Summary updated

Abstract

Each time that an intrusion detection system raises an alert it must make some attempt to communicate the information to an operator. This communication channel can easily become the target of a denial of service attack because, like all communication channels, it has a fixed capacity. If this channel can become overwhelmed with bogus data, an attacker can quickly achieve complete neutralisation of intrusion detection capability. Although these types of attack are very hard to stop completely, our aim is to present techniques that improve alert throughput and capacity to such an extent that the resources required to successfully mount the attack become prohibitive.
BibTeX
@misc{tedesco2003adaptive,
  author = {Gianni Tedesco and Uwe Aickelin},
  title = {{Adaptive Alert Throttling for Intrusion Detection Systems}},
  year = {2003},
  eprint = {1305.7128},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1305.7128},
  howpublished = {Ssrn},
}

Loading related content...

Figure