Committing to a Randomized Query Rule When Attackers Can Poison the Unlabeled Pool

Active learning is the practice of letting a model choose which unlabeled examples a human should label next. It usually works by querying the examples the model is least sure about. In security settings such as network intrusion detection, the unlabeled pool is not neutral: an attacker can inject traffic into it. That turns the model's curiosity into an attack surface. If the learner always asks about the most confusing points, an attacker can cheaply manufacture confusing points and spend the analysts' labeling budget on garbage, while the attacker's real traffic stays unlabeled and unlearned. We propose to treat the choice of which points to query as a leader–follower security game. The defender publicly commits to a randomized query rule over regions of an "uncertainty map." The attacker then decides where to inject, paying a crafting cost that depends on the region.

The map has two axes. The first is committee disagreement, meaning how much a set of models disagree about a point; this tracks what the model does not yet know. The second is irreducible noise, meaning how ambiguous the point is even when the models agree; this tracks what cannot be learned. The central modeling choice is that this game is not zero-sum. The attacker wants wasted queries and an unlearned attack class, and it pays to craft its points. The defender wants accuracy and pays per label. Neither payoff is the negative of the other. We claim that solving the general-sum version gives a query rule that keeps most of the label efficiency of uncertainty sampling on clean data and caps budget waste under an adaptive attacker. We also claim that the zero-sum version, which is the only kind of game existing game-theoretic active learning uses, is measurably too conservative.

Core Claim and Relation to Prior Work

The closest prior work is A Robust Zero-Sum Game Framework for Pool-based Active Learning (RZSG). It casts active learning as a minimax game between a classifier and a selector that reweights the pool. The selector's payoff is the negative of the classifier's, and expected loss is estimated from one model's confidence. Its "adversary" is a fictitious device for robustness. No real attacker exists, no one pays costs, and the model has no notion of committee disagreement. Variational Adversarial Active Learning and Minimax Active Learning also use minimax games, but as training devices inside the learner: a VAE against a discriminator, or an encoder against an entropy-maximizing classifier. Neither models an outside party with its own goals. MultiBreak sits on the other side of the fence. It uses disagreement among safety judges to generate borderline attacks, not to defend a learner against them. Active Inverse Learning in Stackelberg Trajectory Games has a leader choose actions that make competing hypotheses distinguishable. It does this for robot trajectories, not for label acquisition under poisoning. Our proposal differs from all of these in three ways: the adversary is real, its payoff is its own, and the defender's strategy space is defined by the split between what can be learned and what is noise.

Three results motivate the design. First, Lazy Defenders Are Almost Optimal against Diligent Attackers shows two things. In zero-sum security games, a defender who optimizes as if the attacker saw everything loses at most $O(\sqrt{\ln(mdk)/k})$ against an attacker who sees only $k$ samples. In general-sum games, that guarantee breaks down. This is why we must solve the general-sum game rather than borrow a minimax solution. Second, Nonzero-sum Adversarial Hypothesis Testing Games shows that a cost-bearing attacker concentrates on its cheapest effective manipulation. That predicts where injection will land: wherever the defender's rule rewards cheap confusion. Third, Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction finds that choosing median rather than maximal noise-type uncertainty avoids spending effort on outliers. This constrains the design, because any fixed rule of that kind is a fixed target that an adaptive attacker can simply move to.

Research Program

At each labeling round, the defender maintains a small committee: a deep ensemble, or MC-dropout samples as a cheaper variant. For every unlabeled point it computes two quantities. Total predictive entropy splits into a disagreement part, the BALD mutual information studied in A Study of Acquisition Functions for Medical Imaging Deep Active Learning, and a noise part, the expected entropy of individual members. We bin the pool into a small grid of cells on these two axes; a 5×5 grid is the default. These cells play the role of "targets" in a security game.

The defender's strategy is a distribution $x$ over cells. Within the chosen cell it samples a point uniformly. The attacker's strategy is how many points $a_c$ to inject into cell $c$, at a per-point cost $\kappa_c$. That cost is estimated empirically by running point-crafting procedures, such as boundary mixups, label-flipped near-duplicates, and feature-space extrapolation, and measuring how much crafting effort it takes to land a point in each cell. Payoffs are then written as

$$U_D = \sum_c x_c\left[(1-\rho_c)\,g_c - \rho_c\,\beta\right], \qquad U_A = \sum_c x_c\,\rho_c\,\gamma \;+\; \eta\,E(x) \;-\; \sum_c \kappa_c a_c,$$

where the symbols are defined as follows:

  • $\rho_c = a_c/(n_c + a_c)$ is the injected fraction of cell $c$, with $n_c$ its genuine points.
  • $g_c$ is the estimated learning gain from labeling a genuine point in that cell, estimated by looking back at past rounds: the change in validation loss per label drawn from that cell.
  • $\beta$ is the defender's cost for a wasted label.
  • $\gamma$ is the attacker's value for a wasted defender query.
  • $E(x)$ is how poorly the attacker's own attack class is covered by the queries.
  • $\eta$ weights evasion relative to wasted queries.

The attacker pays crafting costs that give the defender nothing, and the defender pays label costs that give the attacker nothing. The game is therefore general-sum. With 25 cells, the strong leader–follower equilibrium can be computed exactly by solving one small linear program per attacker best-response cell. The cost is negligible next to retraining.

The research has three parts. The first is building the defender and attacker simulators on intrusion data. The second is measuring how far the zero-sum simplification, $U_A = -U_D$, moves the equilibrium. The third is testing the lazy-defender question empirically in active learning: when the attacker only learns which of its injected points got labeled over $k$ rounds, is committing to the full-information equilibrium still near-optimal? Lazy Defenders Are Almost Optimal against Diligent Attackers gives no guarantee in the general-sum case, so this is an open empirical question.

Baselines

The naive reference is uniform random sampling. It is surprisingly hard to exploit. An attacker can waste at most its injected fraction of the budget, so it is the natural robustness floor. Our rule must match its waste fraction under attack while clearly beating its accuracy on clean data. All methods share the same backbone, the same per-round label budget, the same retraining schedule, and a committee of the same size wherever a committee is used.

The strongest existing active learning competitor on clean data is BALD with the same committee, following A Study of Acquisition Functions for Medical Imaging Deep Active Learning. By construction it already discounts pure noise, which makes it the most dangerous comparison. We also run RZSG (A Robust Zero-Sum Game Framework for Pool-based Active Learning) with its regularization parameter $\lambda$ tuned on the same validation budget as our method. The file reports only qualitative results for it: on CIFAR-10 it reached "higher accuracy with only 600 labels than competing methods did with 1000 labels." We treat reproducing that ordering over random and margin sampling as a sanity check on our CIFAR-10 pipeline. Further baselines are median-noise selection in the style of Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction and plain entropy sampling.

On the intrusion benchmarks, the fully supervised ceiling is the xIDS-EnsembleGuard result, "average accuracy of approximately 98.8%" across CIC-IDS-2017, UNSW-NB15 and NSL-KDD (xIDS-EnsembleGuard). No reported result was found on Lacuna for active learning under pool poisoning on these datasets, so we establish those numbers ourselves.

Two ablations isolate the key ingredients. The zero-sum ablation solves the identical game with $U_A = -U_D$. The deterministic ablation always queries the defender's single best cell, which removes randomization. The method counts as beating the baselines if it meets two conditions. Under a best-response attacker, it must keep budget waste at or below random sampling's while reaching higher macro-F1 and attack-class recall at every budget. On clean data, it must stay within the run-to-run variance of BALD.

Why not just…?

Why not just use BALD, since it already ignores noise? BALD does discount label-flipped near-duplicates, because committee members agree those points are ambiguous. But points that are cheap to push away from the data, such as extrapolated feature vectors, create disagreement rather than noise, and BALD ranks them first. This is a real threat to the proposal. If extrapolation attacks turn out to be expensive on intrusion features, BALD may be enough. The attacker-cost measurement step and the best-response-against-BALD experiment settle this directly. If BALD's waste stays near random's under its own best-response attacker, we will report that the game adds nothing.

Why not just filter outliers before querying? An outlier filter is a fixed threshold, and a fixed threshold is exactly what a cost-minimizing attacker exploits: Nonzero-sum Adversarial Hypothesis Testing Games predicts it will sit just inside it. We include a filter-plus-BALD baseline and attack it with a best-responding attacker.

Why not just mix uncertainty sampling with some random sampling? This is the most serious simple competitor, because an ε-mixture is itself a randomized commitment. Our equilibrium strictly contains it: ε-greedy is the special case with uniform weight across cells plus a point mass. The question is whether choosing the weights from the costs matters. We sweep ε over a grid and compare against the best ε in hindsight, which is an advantage given only to the baseline. If the best ε matches the equilibrium, the honest conclusion is that the game-theoretic machinery only picks ε.

Why not just use the zero-sum minimax rule, as RZSG does? The zero-sum rule assumes the attacker wants to minimize defender accuracy everywhere. That pushes the defender toward near-uniform querying even when attacks in most cells are too expensive to matter. The zero-sum ablation measures what this conservatism costs in clean-data accuracy.

Proposed Experiments

The main testbed is CIC-IDS-2017, UNSW-NB15 and NSL-KDD. Each starts with a small labeled seed, has per-round budgets, and allows an injection budget of 1–10% of the pool. CIFAR-10 serves as a non-security check that clean-data performance is competitive with the numbers the RZSG paper frames. We use four attackers: a noise injector, a disagreement injector, a median-targeting injector, and a best-response attacker that knows the defender's mixed strategy. We report accuracy, macro-F1, attack-class recall, and fraction of budget wasted, each plotted against labels spent.

Three results decide the idea. First, under the best-response attacker, the equilibrium rule must beat BALD, median selection and the best ε-mixture on attack-class recall at equal waste. Second, the zero-sum ablation must lose clean-data accuracy relative to the general-sum rule when crafting costs are heterogeneous across cells, and the two must converge when costs are uniform; this is a built-in control, since uniform costs make the general-sum structure matter less. Third, we vary how many rounds of feedback the attacker sees, $k$, and measure the gap between committing to the full-information equilibrium and re-solving against a $k$-observation attacker. This tests empirically whether lazy commitment survives in this general-sum setting. We also perturb the estimated costs $\kappa_c$ by up to a factor of two, to show whether the method degrades gracefully when the attacker model is wrong.