Bounded Retained History in a Proxy Flow Replay: A Mixed Predictive Pilot and an Allocation Study

Juan S. SantillanaarXiv 2026 · arXiv

TL;DR

This paper investigates how memory constraints impact the "retained history" used by network security classifiers to detect reconnaissance. The author finds that standard First-In-First-Out (FIFO) buffers lose most historical context to capacity eviction long before time-based expiration occurs, and while source-balanced eviction policies can preserve more diverse history, they do not necessarily lead to a clear predictive advantage in detection.

What Problem Does This Paper Address?

In network security, detecting reconnaissance (like port scanning) usually requires looking at a sequence of events over time—what the author calls "retained history." However, real-world systems have finite memory. When a system reaches its capacity, it must decide which old records to "evict" to make room for new ones.

The author identifies a critical gap: most research assumes an ideal "time window" of history is available, but in practice, the capacity limit often triggers eviction much faster than the time limit. This study asks two questions:

  1. Does adding complex historical features actually improve reconnaissance detection under these constraints?
  2. Can we design better eviction policies to preserve more useful history?

Methods

The study uses an artificial "proxy flow replay" where network records are processed in the order of their timestamps (ltime). The author compared three levels of features:

  • B0: Basic current-flow features.
  • B1: B0 plus simple counters (e.g., source event count).
  • B2: B1 plus five complex "relational" features, such as source out-degree ($O_s$) and service diversity ($V_{sd}$).

To test how memory management affects these features, the author compared several eviction policies (detailed in Figure 3):

  • FIFO: The standard "oldest goes first" approach.
  • SRC (Source-Balanced): Evicts the oldest record from the source currently occupying the most memory.
  • DIV: A policy that prioritizes evicting duplicate relations from the same source.
The five retention arms tested in the allocation study, comparing different record caps and eviction strategies.

The predictive models were built using HistGradientBoostingClassifier with a frozen calibration procedure to ensure the results were not just artifacts of hyperparameter tuning.

Datasets

The study utilizes a curated mirror of the UNSW-NB15 dataset, a common benchmark for network intrusion detection. The author notes several important caveats: the mirror contains significantly fewer rows than the official full collection, and the data was reused across the pilot and the allocation study, which limits the ability to claim "independent confirmation."

Key Insights and Ideas

  • Capacity Trumps Expiration: In a 256-record FIFO buffer with a 300-tick time horizon, the audit found that capacity eviction governed almost everything. The median record lifetime was only 11 ticks—meaning the vast majority of the 300-tick window was lost.
  • The Weighting Paradox: The source-balanced (SRC) policy successfully preserved more history when measured by an "equal-source" metric (giving every IP address equal weight). However, it performed worse than FIFO when measured by "event-weighting" (total events). This means SRC helps track "quiet" scanners but loses more data from "loud" ones.
  • Relational Features are a Mixed Bag: Adding complex history (Model B2) didn't provide a "slam dunk" improvement. While it lowered the false-positive rate, it also lowered the recall for reconnaissance.

Key Results

The predictive pilot (Table 1) showed that the most complex model (B2) actually had a lower AUROC (0.678) compared to the simpler baseline B1 (0.735).

Table 1: Frozen calibrated pilot results. Each stream uses its own previously frozen operating threshold.

In the allocation study, the SRC-256 policy met the author's "materiality rule" for success, outperforming FIFO-256 by a pooled endpoint difference of 0.440738 in terms of equal-source history retention. However, the author emphasizes that this result is specific to the weighting objective and does not prove that detection would improve in a real-world deployment.

Limitations and Future Work

The author is remarkably candid about the study's bounds:

  • Data Reuse: The same development trace was used for multiple stages of the study, which can introduce bias.
  • Proxy Timing: The use of ltime ticks is an artificial convention that may not reflect real-world physical arrival times or detection delays.
  • Lack of Independent Corpus: An attempt to use an external dataset (UWF-ZeekData) was rejected due to quality issues, meaning the findings haven't been validated on a second, independent population.

Future work would require a "defensible population" and an "outcome-untouched evaluation design" to see if the improved history retention of the SRC policy can actually be translated into a more effective, real-time detector.

Summary updated

Abstract

We examine bounded retained history through a frozen predictive pilot and a subsequent allocation study on a curated UNSW-NB15 flow mirror. Both use an artificial release-order replay based on raw ltime ticks, with equal-tick records querying a shared prior-group state. In the pilot, the model with added retained relational features had lower reconnaissance recall than the current-flow-plus-counters baseline (43.66% versus 47.84%) and lower benign false-positive rate (3.63% versus 6.68%) at separately frozen operating thresholds, together with lower AUROC. This mixed result did not establish a predictive advantage. A development-only audit subsequently found that capacity eviction dominated expiration in the 256-record, 300-tick FIFO state, with a median completed record lifetime of 11 ticks. A protocol specified before its single execution, but after the pilot and post-hoc mechanism audit, compared five label-blind retention arms on the already-used development trace. Its primary equal-source endpoint averaged retained/reference ratios for service-expanded source out-degree and source-destination service diversity. Source-balanced retention at 256 records exceeded FIFO-256 by a pooled endpoint difference of 0.440738 and met the prespecified materiality and subblock rule. Crucially, the event-weighted ratios for both quantities decreased relative to FIFO-256: the result is specific to the equal-source weighting objective and does not establish uniformly improved retention. The study supplies neither evidence of detection improvement nor an explanation of the pilot result. Reused data, concentrated reconnaissance support, proxy timing, and the absence of a qualified independent corpus bound the interpretation.
BibTeX
@misc{santillana2026bounded,
  author = {Juan S. Santillana},
  title = {{Bounded Retained History in a Proxy Flow Replay: A Mixed Predictive Pilot and an Allocation Study}},
  year = {2026},
  eprint = {2611.02123},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02123},
}
Figure