Forecasting What a Syscall Profile Is Still Missing

Dynamic analysis learns what a program does by watching it run. That means it can only report behavior that actually happened while it was watching. Container sandboxing shows the cost of this clearly. Tools that build a system-call allowlist, meaning the set of operating-system requests a container is permitted to make, trace the container for a while and then block everything they did not see. Any request that happens only under an unusual configuration or an uncommon workload is missed, and the resulting policy later breaks legitimate behavior in production. BEACON, which records system calls through low-overhead kernel tracing, found that varying Docker options exposed 11% more required system calls and varying workloads exposed another 22% (BEACON). BEACON finds these by brute force, enumerating perturbations one after another. It then reduces each trace to a flat set, so the order and phase structure of the trace are thrown away.

We propose to predict that missing behavior from a single short, cheap trace. A forecaster reads the trace as a hierarchy: container lifecycle phases, then request-level episodes, then individual system calls. For each call the profile has not yet seen, it predicts the probability that the call will be needed. The model is interpretable by construction because each prediction is a sum of contributions from identifiable trace segments. An auditor can therefore see which part of the observed run implies that, say, epoll_pwait or clone3 is still missing, and which configuration or workload perturbation is likely to trigger it. Profiling then becomes a targeted loop that stops early once the predicted missing mass is small, instead of an exhaustive sweep.

Core Claim

The claim is that the temporal and phase structure of an observed trace carries information about behavior that has not happened yet. That information is not recoverable from the flat set of observed calls, from frequency counts, or from what similar images do. If this is true, a segment-additive hierarchical forecaster should beat set-based predictors at identifying missing system calls on held-out images. It should also reach BEACON-level policy quality with a fraction of BEACON's perturbation runs. If it is false, a flat co-occurrence model will match the forecaster. We treat that outcome as a real possible result, not a straw man.

Each direction plays a different role here. The coverage gap in dynamic analysis is the problem being solved, and container profiling is the testbed. Hierarchical temporal modeling supplies the encoder. Interpretable-by-design forecasting supplies the constraint on the output, which forces every prediction to be traced back to trace segments.

Relation to Prior Work

Three papers motivate the problem. BEACON shows the environment gap and closes it by enumerating perturbations (BEACON). We reuse its tracing and perturbation space, but we predict which perturbations matter instead of trying them all. Going Native shows that short observation windows in Android native-code sandboxing produce under-approximated profiles that block legitimate behavior. That false-block failure is the downstream harm we measure. Towards locating execution omission errors forces skipped code to run by flipping branch outcomes, at an 18.3x to 154.9x slowdown. Our approach is the opposite trade: we predict the omitted behavior cheaply and spend active execution only where the prediction says it is worth it.

The closest existing work is Efficient Forecasting of Task Failures in LLM Agents through Adaptive Fault Injection. It forecasts agent success or failure from a partial trace and fills gaps in rare states by injecting faults chosen with a bandit. It differs from our proposal in three ways. It predicts one scalar outcome, while we predict a set of specific unobserved behaviors. It reads traces as flat text with a black-box 8B model, while we use a small structured model whose outputs decompose over segments. And it does its perturbation offline during data curation, while we use predictions to choose perturbations at profiling time. That paper also reports that forecasters transfer poorly across agent policies. This is a warning we take seriously: our held-out image families are designed to test the same kind of transfer.

Interpretable Early Failure Detection via Trace Checking learns human-readable temporal-logic rules over telemetry using evolutionary search. It shows that trace-level interpretability is useful, but it works on continuous sensor streams and learns no multi-scale representation. Feedback-driven dynamic invariant discovery uses symbolic execution to refute invariants that were wrongly inferred from sparse coverage. That is a coverage-gap feedback loop too, but it treats traces as unordered sets of states and forecasts nothing.

The encoder design draws on video work. Temporal Convolutional Networks for Action Segmentation and Detection shows that encoder-decoder temporal convolutions beat flat recurrent models on long sequences (64.7% vs 55.7% accuracy on 50 Salads) and avoid rapid label flickering. StreamForest merges history into a tree of events using content similarity and temporal distance, and keeps 96.8% of its accuracy under an 8x token reduction. We borrow its merge rule to build segments from system-call windows. Lavanchy et al. predict both coarse phases and fine steps in surgical video. They also show a phase F1 collapse from about 90% to about 33% across hospitals with different procedural styles. We read that as a constraint: a hierarchy learned on one family of images may not transfer to another.

On interpretability, N-BEATS gets transparency by making the forecast an additive sum of constrained block outputs. We keep the additive structure. We drop the polynomial and Fourier bases, because they have no meaning for discrete system-call events. DA-RNN and STAM attribute forecasts through attention over input variables and time steps. The file itself notes that attention shows where a model looks, not what drives its output. We therefore test faithfulness directly instead of assuming it.

Research Program

Data construction. Following BEACON's setup, we trace the 167 official Docker images with eBPF across the cross-product of Docker options and workload profiles, including the Yahoo! Cloud Serving Benchmark workloads BEACON used. For each image, the union of system calls across all runs defines its reference profile. The model's input is a single trace under default options and a short workload window. The target is the set of reference-profile calls absent from that trace, with each missing call labeled by the perturbations that exposed it. We split by image family (databases, web servers, language runtimes, and so on) as well as at random. The family split reproduces the cross-institution setting from the surgical-video work.

Model. The trace is first cut into fixed windows of calls. Windows are then merged into episodes with a StreamForest-style penalty that trades content similarity against temporal distance, and episodes are merged again into lifecycle phases. A dilated temporal convolution encodes each segment $g$ at every level into a vector $h_g$. For a candidate call $s$, the predicted log-odds that $s$ is required but unobserved are

$$\operatorname{logit} p(s \mid x) = b_s + \sum_{g \in \mathcal{G}(x)} f_s(h_g),$$

where $x$ is the trace, $\mathcal{G}(x)$ is the set of segments at all hierarchy levels, $b_s$ is a per-call prior, and $f_s$ is a small shared network with a per-call output head. Because the log-odds are exactly a sum over segments, each segment's contribution is its explanation, and no post-hoc attribution method is needed.

A second head predicts, for each missing call, which perturbation class will expose it. At profiling time, the system runs only the perturbations whose predicted yield is high. It stops once the predicted total missing mass falls below a threshold.

Interpretability test. We test whether explanations are faithful by deleting or replacing the top-attributed segment and checking whether the prediction drops as much as the attribution says it should. We compare this against attention weights from a flat sequence model under the same deletion test. We also run a small audit: for a sample of predictions, a security engineer checks whether the attributed segment plausibly relates to the predicted call, for example a network-setup phase implying socket-option calls.

Baselines

The naive reference point is to deploy the observed default-trace profile as-is. This is dynamic profiling with no perturbation, the practice that Going Native and BEACON show under-approximates. It sets the floor for missing-call recall, which is zero, and for production false-block rate.

The strongest existing method is full BEACON: exhaustive environment and workload variation followed by its security and functionality scoring. BEACON reports mitigating 85.3% of 45 tested kernel CVEs across 167 images, against 58.6% for the static tool Confine, with tighter policies in 80% of cases (BEACON). We rerun it on our images as the quality ceiling and the cost reference. Beating it does not mean exceeding 85.3%. It means matching its CVE mitigation within a small margin and its production false-block rate while running substantially fewer perturbation traces. We will report the whole curve of cost against coverage, not a single operating point.

The strongest competitors for the prediction task itself are set-based. The first is an unseen-mass estimator in the Good-Turing style, which estimates how much behavior is still missing from how many calls were seen only once. The second is a co-occurrence model: a factorization over images' observed call sets plus image metadata, predicting which calls are missing from what similar images use. For this prediction task, no reported result was found on Lacuna for either method, so all such numbers will come from our own runs.

The flat temporal competitor is a STAM-style causal attention LSTM (STAM) and a same-size flat Transformer, both over the raw call sequence. All learned models get identical training images, the same parameter budget within a factor of 1.5, the same hyperparameter search budget, and the same held-out family split.

Two ablations isolate the proposal's key ingredients. The first replaces the learned hierarchical segments with fixed-length windows, which tests the hierarchy. The second removes the additive constraint by concatenating segment encodings into an unconstrained classifier, which measures the accuracy cost of built-in interpretability. The hierarchy claim holds only if the full model beats the fixed-window ablation and the co-occurrence model on missing-call recall at a fixed false-positive rate on held-out families.

Why not just…?

Why not just estimate the missing mass from frequency counts? A Good-Turing-style estimate is cheap and principled, and it might predict when to stop profiling as well as our model does. It cannot say which calls are missing or which perturbation would reveal them, and both are needed to write a policy and to target the next run. We still run it head-to-head on stopping decisions. If it matches our stopping curve, we will report that the forecaster's value lies only in naming calls and perturbations.

Why not just predict missing calls from what similar images use? This is the most serious threat. BEACON's own success with flat sets suggests that order may not matter much. If images running the same runtime share their dormant calls, a co-occurrence model could capture most of the signal with no temporal model at all. The comparison on held-out image families settles it. If the hierarchical model does not beat co-occurrence there, the core claim fails and we report that result. We expect temporal structure to matter most for images whose metadata is uninformative, and we will analyze those images separately.

Why not just take the union with a static analysis? Static tools cover all paths, but they over-approximate. Confine mitigated only 58.6% of CVEs against BEACON's 85.3% (BEACON) because its allowlists are too loose. We include "observed plus static" as a baseline. We expect it to have high recall but loose policies. Our model is judged on the precision it adds at matched recall.

Why not just use attention weights from a flat forecaster as the explanation? Attention models such as DA-RNN are accurate, but their weights are not guaranteed to reflect what drives the output. The deletion test compares attention-based and additive attributions under the same intervention. That shows directly whether the additive constraint buys faithfulness, and the second ablation shows what it costs in accuracy.

Decisive Checks

The main result is recall of missing calls at a fixed precision on held-out image families. This compares the full model against co-occurrence, Good-Turing ranking, flat attention models, and the two ablations. Deployment metrics come next. For each method we generate policies, replay held-out workloads, and report the false-block rate, the CVE mitigation rate on BEACON's 45 CVEs, and the number of perturbation traces used compared with full BEACON. The faithfulness check is the prediction drop under deletion of top-attributed segments compared with deletion of random segments.

The project fails on its central claim if co-occurrence matches the hierarchical model on held-out families, or if the additive model's attributions are no more faithful than attention. Either failure would still be informative, because it would tell practitioners that cheap set-based completion is enough to close much of the environment gap.