Many models that make high-stakes decisions read discrete inputs: merchant codes, network log verbs, malware signature bits, DNA letters. Two threats hit them. An attacker can swap a few categories at test time to flip a decision, which is an evasion attack. An attacker can also plant a backdoor: poisoned training rows carrying a hidden trigger, such as a particular combination of category values, that later forces a chosen output while the model looks normal on clean data. The standard fix for evasion is adversarial training, meaning training on worst-case perturbed inputs. In images, this fix makes backdoors worse. On the Trade-off between Adversarial and Backdoor Robustness found that adversarially trained vision models treat a trigger that perfectly predicts the target label as an especially reliable "robust feature." Backdoor success rose from 64.1% to 99.9% on CIFAR-10 and from 3.9% to 65.4% on ImageNet. Nobody has checked whether the same thing happens when categorical models are hardened. That matters because tabular backdoors already defeat every classical defense tested against them, as shown by CatBack.
This note proposes a measurement study and a single-pass defense built on what the study predicts. The prediction is that the harm depends on how the model is hardened. Attack-based categorical adversarial training should amplify sparse categorical backdoors. An attack-free regularizer that penalizes reliance on a few features should partly counteract them. The per-row version of that same "few features decide" signal, combined with early-training confidence, should let a defender remove poisoned rows before training. That defense would handle both threats at roughly the cost of one ordinary training run and would avoid the robustness-accuracy tax.
Core Claim
The "better student" effect is not a property of robustness in general. It comes from attack-based training, which removes brittle features without discouraging concentrated reliance on a small feature subset. A CatBack trigger is exactly such a subset, because its Elastic-Net optimization makes it sparse by design. Attack-free Evaluating and Enhancing Adversarial Robustness on Categorical Data introduced IGSG, a penalty on the spread of integrated-gradient attributions across features plus a smoothed-gradient curvature term, and showed that over-reliance on a few features drives categorical vulnerability. We therefore predict three outcomes. First, probabilistic adversarial training (PADVT, from Probabilistic Categorical Adversarial Attack and Adversarial Training) will raise backdoor success on tabular data, especially for weak, low-rate poisons that barely work on a standard model. Second, IGSG regularization will raise it less or lower it, but only partly. Its penalty is averaged over the batch, and poisoned rows are a few percent of the data, so the penalty on them is diluted. Third, scoring each training row by how concentrated its attributions are, and pairing that with cumulative prediction entropy, will separate poisoned from clean rows even in the case where entropy alone fails.
The third claim addresses a tension in the file. Anti-Backdoor Coreset Selection via Cumulative Entropy (ABCS) removes poisoned samples because the model learns triggers early and confidently, with low entropy. That result was shown only on images. The "robust feature" view suggests that under hardening, hard clean rows may also become confidently fit, which would blur the entropy gap. Attribution concentration is a second axis that should stay informative, because a sparse trigger is concentrated whether or not the model is confident.
Research Program
The first part is the measurement study. We would reimplement CatBack's poisoning: map categories to frequencies, optimize an Elastic-Net-regularized universal trigger, then map the values back to valid categories. We would poison five datasets at rates chosen so that the standard model's attack success spans weak to strong. On the same poisoned data, we then train four kinds of models. The first uses standard training. The second uses PADVT. The third uses Cat-PGD adversarial training from Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings, which relaxes one-hot vectors inside their convex hull and projects them back to valid categories. The fourth uses IGSG regularization. For each, we record how backdoor success changes relative to standard training and how clean and robust accuracy change. This settles whether the vision finding carries over to discrete inputs and whether the kind of regularizer matters.
The second part is the defense, which we call attribution-entropy selection. We warm-start a model for a few epochs of ordinary training. For each row $i$, we accumulate its prediction entropy $H_t(x_i)$ over warm-up epochs $t = 1..T$, as ABCS does. Alongside it, we compute an attribution concentration $C_t(x_i)$: the share of the row's total absolute integrated-gradient attribution carried by its top-$k$ features. The row's suspicion score is
$$s_i = \sum_{t=1}^{T} \big( C_t(x_i) - \lambda\, H_t(x_i) \big),$$
where $\lambda$ balances the two terms and is set on a held-out clean split. We drop the highest-scoring rows, keeping a coreset whose size is chosen by a clean-validation accuracy tolerance rather than fixed at ABCS's roughly 55%. We then train the final model on that coreset with IGSG regularization so it also resists evasion. There is no inner attack loop at any point. The integrated-gradient pass during warm-up is the only extra cost, and IGSG's reported 17 to 240 times speedup over attack-based training bounds how expensive this kind of computation is relative to adversarial training.
The robustness-accuracy trade-off acts as a hard constraint on this design. TableShift found that robustness methods on tabular data often shrink the in-distribution-to-shift gap only by lowering in-distribution accuracy, and they lose to tuned gradient-boosted trees. Discarding data from small tabular datasets could cause exactly that loss. We therefore report clean accuracy against a tuned LightGBM trained on the full clean data, and we treat any defense that loses more than a point of clean accuracy against it as failing.
Baselines
The naive reference point is standard training on the poisoned data with no defense. By construction it has high clean accuracy and high backdoor success: CatBack reports near-100% attack success on the Adult, Bank Marketing, CovType, Credit and HIGGS datasets. Every method uses the same poisoned splits, the same architectures (MLP, FT-Transformer and TabTransformer, the neural models benchmarked in TabAttackBench), and the same hyperparameter search budget, counted in trials. Wall-clock time is logged on identical hardware.
The strongest existing backdoor defense to compare against is ABCS applied directly to tabular data, with its cumulative-entropy score and unlearning step. It is the strongest competitor because it already achieves low overhead and preserves clean accuracy, and it reports defense effectiveness above 95% on CIFAR-10 with post-defense attack success near 1%. Those are vision numbers. We found no reported result on Lacuna for ABCS, or for any successful defense, on tabular categorical backdoors. We also run the classical defenses CatBack tested, Spectral Signatures and Fine-Pruning, to confirm that they fail; CatBack reports that Spectral Signatures missed poisoned rows in over 75% of cases.
For evasion, the strongest baselines are PADVT and IGSG regularization. The file gives PADVT's IMDB error under attack as roughly 33%, down from roughly 92%, and IGSG's adversarial accuracy on PEDec as 86.5% against 74.3% for the next best baseline. We evaluate robust accuracy under two attacks, PCAA and Cat-PGD, so that no defense is scored only against the attack it trained on. We would count the proposal as beating the baselines if it meets all of the following at once. Its backdoor success must be at least as low as ABCS's on tabular data. Its robust accuracy must be within two points of the better of PADVT and IGSG regularization. Its clean accuracy must be within one point of standard training. Its wall-clock time must be below PADVT's.
Two ablations isolate the key ingredient. The first uses entropy only, which is ABCS followed by IGSG regularization. The second uses concentration only. If entropy alone matches the full score after hardening, the concentration term is not doing any work.
Why not just…?
Why not just use gradient-boosted trees? Trees are the right clean-accuracy yardstick, and TableShift shows they beat robust neural networks under natural distribution shift. But they are not a defense. CatBack backdoors tree models and Google's AutoML service with near-100% success. Robust tree training took nearly three hours on Credit, compared with three minutes for the embedding transfer in Transferable Adversarial Robustness. We include LightGBM, both plain and fed our selected coreset, so that it serves as a yardstick and a possible user of the defense rather than a rival.
Why not just run ABCS on tabular data? This is the real threat. If cumulative entropy separates CatBack rows as cleanly as it separates image triggers, the defense contribution reduces to "ABCS plus IGSG regularization." The measurement study remains useful, but the new selection score would not be. The entropy-only ablation settles this directly. We will also report the setting where entropy should fail: rows scored with a warm-up that includes hardening, and low poison rates. The concentration term has to earn its place there.
Why not just train with IGSG regularization and skip data selection? The batch-averaged attribution penalty might already suppress a sparse trigger. If IGSG regularization alone drives backdoor success near zero, the core claim is confirmed and the simpler recipe wins. We expect dilution at realistic poison rates, and the IGSG-only arm of the measurement study tests exactly that.
Why not just filter with an outlier detector? CatBack reports that its Elastic-Net triggers bypass popular outlier detection, because they are sparse and stay close to the data distribution. We include an isolation-style detector trained on raw rows as a sanity baseline, without expecting it to compete.
Decisive Checks
Three results would decide the proposal. The first is whether PADVT raises backdoor success over standard training at weak poison rates on at least three of the five CatBack datasets. That would carry the vision finding, including the MNIST jump from 17.2% to 67.2%, into discrete inputs. A negative result would be publishable and would redirect the work. The second is whether IGSG regularization increases backdoor success significantly less than PADVT, which tests whether the kind of regularizer matters. The third is whether, on hardened warm-ups, the full score separates poisoned rows better than entropy alone, measured by area under the precision-recall curve for detection, while keeping clean accuracy within a point of tuned LightGBM. For discrete-only inputs, we repeat the first two checks on Splice and PEDec, poisoning them with a sparse categorical trigger built the CatBack way.
References
- On the Trade-off between Adversarial and Backdoor Robustness
- CatBack: Universal Backdoor Attacks on Tabular Data via Categorical Encoding
- Attack-free Evaluating and Enhancing Adversarial Robustness on Categorical Data
- Probabilistic Categorical Adversarial Attack and Adversarial Training
- Anti-Backdoor Coreset Selection via Cumulative Entropy
- Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings
- Benchmarking Distribution Shift in Tabular Data with TableShift
- TabAttackBench: A benchmark for adversarial attacks on tabular data