Data pruning means training on a small, carefully chosen subset of a dataset instead of the whole thing. It is attractive because many examples are redundant. The most effective pruning scores tend to keep the "hard" examples: those the model keeps forgetting, stays uncertain about, or that carry high influence. Influence here means the estimated effect of one training example on the model's parameters or loss. Backdoor attacks plant a small set of poisoned training examples that teach the model a hidden rule: behave normally, but output the attacker's label whenever a trigger pattern appears. These two facts collide. Data-Efficient Backdoor Attacks and its follow-up Explore the Effect of Data Selection on Poison Efficiency in Backdoor Attacks show that attackers get the most out of their poisons by choosing the frequently forgotten, hard ones. Those are exactly the examples a difficulty-based pruner would keep. Aggressive pruning may therefore concentrate the poison, raising its share of the coreset from 1% to several percent.
We propose a pruning score designed to resist this. A poisoned group differs from hard clean examples in one way: its members all teach the model the same thing. Each hard clean example is hard for its own reason. Poisons are useful to the attacker only because they share one trigger-to-label rule. We therefore score clusters of examples by how much more influence they exert together than separately. We call this their influence coherence. We remove high-coherence clusters and then prune the remainder for utility. The goal is a training-time pruner that keeps 20–30% of the data, rather than the ~55% of the best existing anti-backdoor coreset, and still defeats attacks that were deliberately made hard to learn.
Core Claim
For training examples $i$, let $g_i$ be the gradient of the loss with respect to the parameters. We center it by subtracting the mean gradient of examples sharing the same label, so that the ordinary pull toward the class does not count as coherence. We then compress it with a random projection. Let $H$ be a damped curvature (Gauss-Newton) matrix in that projected space. For a cluster $C$ of examples, define the coherence ratio
$$\kappa_C = \frac{\left\| \sum_{i \in C} g_i \right\|_{H^{-1}}^2}{\sum_{i \in C} \| g_i \|_{H^{-1}}^2}, \qquad \|v\|_{H^{-1}}^2 = v^\top H^{-1} v .$$
The numerator is the group's influence on its own loss, the classical influence formula applied to the group as a whole. The denominator sums the members' individual self-influences. The ratio $\kappa_C$ is near 1 when members push in unrelated directions and approaches $|C|$ when they push identically. This range follows directly from expanding the squared norm of the sum.
The claim is that backdoor clusters have much larger $\kappa_C$ than clean clusters of the same size and label. The gap appears early in training, because the trigger is a shortcut learned in the first epochs. Hard-to-learn poisons such as Adaptive Blend or forgetting-selected poisons can raise each member's loss and entropy. They cannot erase the shared direction without the model failing to learn a single trigger rule. The prediction is falsifiable: if $\kappa_C$ does not separate A-Blend or FUS-selected poisons, the idea fails.
Prior work plays three roles here. Clustered Influence Functions supplies the machinery and the hint. It caches curvature solves for gradient-cluster centroids and observes that averaging within clusters amplifies coherent backdoor signals. It does this, however, as a query oracle on a fixed trained model, and it needs a suspicious query. We use the amplification itself as a query-free score, compute it across early checkpoints, and place it inside a pruning loop.
Delta-Influence motivates the influence route and constrains it. Plain EK-FAC influence attribution reached only 2.9% precision on CIFAR-10 under a frequency trigger, against 13.3% for Delta-Influence. Per-sample influence is too noisy, which is why we work at the group level. Delta-Influence also needs a confirmed poisoned test input; we need none. Backdoor Learning Curves ties the backdoor's initial learning slope to influence functions. This supports the idea that influence measures backdoor uptake, but that paper offers diagnosis, not selection.
Research Program
The first step is diagnostic. On BackdoorBench we will measure poison enrichment: the fraction of poisons in the retained set relative to the original poisoning rate. We will do this for standard pruning scores at retention levels from 10% to 50%. The scores are forgetting counts, cumulative entropy, per-sample self-influence, and influence on a clean validation loss. The attacks are random-selection poisons and FUS-selected poisons. This establishes whether pruning actually amplifies backdoors, which the file implies but does not measure.
The second step builds the pruner. We train for a short warm-up, which ABCS also uses. At several early checkpoints we compute per-example gradients restricted to the last blocks, project them to a few thousand dimensions, and cluster them within each label using k-means. We solve the inverse-curvature products only on cluster sums, reusing the caching strategy of Clustered Influence Functions. We average $\kappa_C$ over checkpoints, which partially addresses the static-checkpoint weakness of influence functions. Clusters whose coherence is far above the per-label null distribution are discarded. The null distribution comes from clusters built on label-permuted subsets. Among the survivors we then select a utility coreset, either by influence on clean validation loss or by entropy. Finally, a fresh model is trained on that coreset.
The third step is adversarial. We craft poisons that explicitly penalize their own gradient alignment. This can be done with sample-specific triggers, with blending strength varied per sample, or with a poison-crafting loss that adds $\kappa$ as a penalty. We then measure the trade-off between lowering coherence and keeping attack success rate (ASR). If attackers can drive $\kappa$ down cheaply while keeping ASR high, the method is broken, and we will report that.
Baselines
The naive reference points are full-data training with no defense, and random pruning to the same retention level. These fix the clean accuracy we must preserve and the ASR we must eliminate.
The strongest pruning-based competitor is Anti-Backdoor Coreset Selection via Cumulative Entropy. It keeps high cumulative-entropy samples and adds label-smoothing unlearning. The paper reports a defense effectiveness rate above 95% on CIFAR-10, above 97% on Tiny-ImageNet, and ASR near 1%, but with a coreset of about 55%. It also reports that it did not fully neutralize Adaptive Blend. We will run it at its native retention and at our matched lower retentions. It will use the same warm-up epochs, architecture, and final-training schedule, with its threshold tuned on the same budget of clean validation data as ours. Beating it means one of two outcomes. Either we match its ASR and clean accuracy at 20–30% retention, or we drive A-Blend ASR substantially lower at equal retention.
The strongest training-time defense overall is Progressive Poisoned Data Isolation (PIPD). It is not a pruner: it isolates poisons progressively through contrasting convolutional channels and trains on all benign data. On CIFAR-10 it reports an average ASR of 0.43%, clean accuracy of 94.5%, poison-detection true-positive rate of 99.95%, and false-positive rate of 0.06%. We do not expect to beat its clean accuracy while training on a quarter of the data. The fair comparison is clean accuracy and ASR per unit of training compute, plus transfer to a transformer, where PIPD's channel heuristic has no direct analogue.
We also compare against COLLIDER, the closest gradient-geometry coreset method. It keeps centrally located gradients plus a local-intrinsic-dimensionality term, and reports 4.80% ASR with 80.66% clean accuracy on CIFAR-10 under BadNets. Matching its ASR with clearly higher accuracy would count as beating it.
Two ablations isolate our key ingredient. The first replaces $H^{-1}$ with the identity, giving plain gradient-alignment coherence. The second scores clusters by group self-influence alone, without dividing by the individual terms. If the identity variant matches the full method, the contribution is coherence rather than influence, and we will say so.
Why not just…?
Why not just use cumulative entropy and shrink the coreset? Entropy is a per-sample difficulty signal, so it is exactly what an adaptive attacker manipulates. ABCS itself reports incomplete defense against A-Blend. FUS-style selection pushes poisons toward the uncertain, forgettable end that entropy-based pruning keeps. Our diagnostic experiment measures whether poison enrichment under entropy pruning grows as retention shrinks. If it does not, this alternative is a real threat and our advantage reduces to the A-Blend case.
Why not just cluster activations or gradients and drop small, pure clusters? This is the strongest cheap alternative, and it is close to the identity-curvature ablation. The difference is that raw gradient norms in deep networks are dominated by a few high-curvature directions. Clean hard examples can therefore look aligned simply because they share those directions. Dividing by individual self-influence and preconditioning by curvature should correct for this. Whether it actually does is decided by the ablation, run on matched cluster sizes. We treat a tie as a likely outcome, not a remote one.
Why not just prune with any standard method and then run a post-hoc defense such as Fine-Pruning or unlabeled-data distillation? Explore the Effect of Data Selection on Poison Efficiency in Backdoor Attacks reports that FUS-selected poisons resist Fine-Pruning. Backdoor Cleansing with Unlabeled Data reports failure at high effective poisoning ratios, which pruning-induced enrichment could produce. We will run this two-stage pipeline at matched retention to test it directly.
Proposed Experiments
All runs use the SCLBD/BackdoorBench codebase. Datasets are CIFAR-10, GTSRB, and Tiny-ImageNet. Models are PreAct-ResNet18 and EfficientNet-B3, chosen because BackdoorBench documents defenses failing to transfer between them, plus DeiT-Tiny on an ImageNet subset following the CiF setup. Attacks are BadNets, Blended, WaNet, A-Blend, the Witches' Brew clean-label attack used in Delta-Influence, FUS-selected poisons, and our coherence-penalized adaptive attack. Poisoning ratios run from 0.5% to 10%.
We report clean accuracy, ASR, defense effectiveness rate, poison enrichment in the coreset, area under the ROC curve of $\kappa_C$ for flagging poisoned clusters, and wall-clock cost including gradient and curvature computation. For clean-data pruning accuracy at our retention levels, we found no CIFAR-10 result reported on Lacuna. We will therefore establish this reference ourselves with random and entropy pruning on unpoisoned data.
The decisive checks are four. First, $\kappa_C$ separates A-Blend and FUS poisons from clean clusters at the warm-up checkpoints. Second, the coherence filter removes the poison enrichment seen with standard pruners. Third, at 25% retention our pipeline matches ABCS's ASR at no worse clean accuracy. Fourth, the coherence-penalized attack must give up substantial ASR to evade the filter. Failure of the first check ends the project.
References
- Data-Efficient Backdoor Attacks
- Explore the Effect of Data Selection on Poison Efficiency in Backdoor Attacks
- Clustered Influence Functions
- Delta-Influence: Unlearning Poisons via Influence Functions
- Backdoor learning curves: explaining backdoor poisoning beyond influence functions
- Anti-Backdoor Coreset Selection via Cumulative Entropy
- Progressive Poisoned Data Isolation for Training-Time Backdoor Defense
- COLLIDER: A Robust Training Framework for Backdoor Data
- Backdoor Cleansing with Unlabeled Data