Classifiers used in particle physics are trained on simulation and applied to real detector data. When data samples become huge, the dominant error is no longer counting noise. It is the residual systematic uncertainty: the leftover mismatch between simulation and reality in detector response, shower modeling, or background shape. The usual machine-learning fix is to make the whole classifier insensitive to a nuisance variable, either with an adversary that tries to recover the nuisance from the output or with worst-case input perturbations. That fix is blunt. Most physical systematics damage only a few class boundaries. Shower-generator differences matter mostly where gluon-rich jets are involved. Track-resolution mismodeling matters mostly for bottom-versus-charm separation. Mass sculpting, where a cut on the classifier distorts the background mass spectrum, matters for signal-versus-QCD-background but not for telling a Z boson from a Higgs boson, a distinction that depends on mass. A global penalty spends discrimination power on boundaries that never needed protecting.
We propose to break a multi-class tagger into its class pairs and apply each systematic constraint only to the pairs it actually affects. Pairwise (one-versus-one) classification trains one binary decision for every pair of classes and combines them afterward. The concrete system is a shared jet encoder with one small binary head per class pair. Each head carries only the nuisance penalties that a measured "sensitivity map" assigns to it. The heads are then combined into a single probability vector with a pairwise-coupling solve. The claim is testable and could fail. We expect pair-scoped constraints to match a global adversary's robustness on the affected boundaries while keeping close to nominal performance on unaffected ones, a better trade-off than any single global penalty strength can reach.
Core Claim
The open question is not whether one-versus-one beats a single softmax model on clean data; Interclass Interference Suppression in Multi-Class Problems already argues this on small UCI datasets. The question is whether where a systematic constraint is applied matters as much as how strongly.
Existing robustness methods all act globally. Decorrelated jet substructure tagging using adversarial neural networks decorrelates a binary signal-versus-QCD tagger from jet mass. It accepts slightly worse nominal separation in exchange for 3.67σ instead of 2.82σ discovery significance at 50% background systematic. Improving Robustness of Jet Tagging Algorithms with Adversarial Training trains a three-class flavor tagger with a single perturbation budget across all classes. It reads off bottom-vs-light and charm-vs-light as ratios of the softmax outputs, but never asks whether different systematics hit different pairs. Both papers supply techniques we reuse. Neither scopes them.
Our hypothesis is that systematics in multi-class jet tagging are approximately sparse over class pairs, and that exploiting this sparsity moves the robustness–performance frontier outward. Systematics of quark/gluon tagging motivates the sparsity assumption. It shows that seven parton-shower generators disagree by nearly a factor of two on quark/gluon separation, driven by nonperturbative effects such as color reconnection. That is exactly the kind of uncertainty that should be localized to gluon-involving boundaries.
Research Program
The first deliverable is a diagnostic: a systematic sensitivity map for the 10-class JetClass benchmark introduced in Particle Transformer for Jet Tagging. We train an unconstrained tagger and compute every pairwise score. For each nuisance variation, we measure how much each pair's score distribution and background rejection shift between nominal and varied samples. We study four nuisance variations:
- a constituent energy-scale shift, applied separately to charged and neutral particles;
- smearing of track impact parameters, which are the displaced-track variables that drive heavy-flavor identification;
- an alternative parton-shower generator for a regenerated subset (we will produce Herwig-showered samples, since JetClass ships with a single generator configuration);
- jet-mass sculpting, measured on the QCD class.
This yields a 45-by-4 matrix. Whether that matrix is actually sparse is the first falsifiable result. If every nuisance moves every pair substantially, the premise fails and we stop there.
Given the map, training uses a Particle Transformer encoder shared across all pairs, with 45 lightweight binary heads. The loss is
$$ \mathcal{L} = \sum_{i<j} \Big[ \ell_{ij}\big(s_{ij}\big) + \lambda \sum_{\nu} M_{ij,\nu}\, D_\nu\big(s_{ij}\big) \Big], $$
where:
- $s_{ij}$ is the head output for pair $(i,j)$, evaluated only on jets of classes $i$ and $j$;
- $\ell_{ij}$ is binary cross-entropy;
- $M_{ij,\nu}\in\{0,1\}$ is the sensitivity map for nuisance $\nu$;
- $D_\nu$ is a nuisance penalty. For mass, this is the adversarial decorrelation of the mass-decorrelation paper. For the energy-scale, impact-parameter, and generator variations, it is a two-sample discrepancy between the head's scores on nominal and varied copies of the same jets.
A shared encoder keeps the cost close to a single model, which answers the quadratic-cost objection to one-versus-one. It also creates a real risk: penalties on some heads can leak through the shared encoder into others. We treat that risk as an experimental variable by also testing per-pair encoder adapters.
Two aggregation methods combine the pairwise heads into a class probability vector. The first is the Kullback–Leibler-minimizing coupling solve of Multi-Classification using One-versus-One Deep Learning Strategy with Joint Probability Estimates, which turns pairwise probabilities into one consistent vector by solving a linear system. That paper worked on clean image and ECG data and turned off uncertain classifiers using distance to the decision boundary, a rule that may not hold up under simulation mismatch. The second is a gated variant that weights each head by the aggregated posterior mass on its two classes. This targets the "irrelevant classifier" problem, where a Higgs-vs-top head is asked about a W jet, and that problem may get worse under shift.
We deliberately do not adopt pairwise difference learning from Uncertainty Quantification in Pairwise Difference Learning for Classification. Its pairing over instances grows with the square of the number of jets, which is infeasible at 100 million jets. It also has no notion of a nuisance parameter.
Baselines
The naive reference point is an unconstrained Particle Transformer with standard multi-class cross-entropy, trained on nominal JetClass, with pairwise scores read off as $p_i/(p_i+p_j)$. On clean data this is the strongest model. The ParT paper reports 86.1% JetClass accuracy, against 84.4% for ParticleNet. Any proposal that loses substantial clean accuracy must justify the loss through robustness.
The strongest existing robustness method is the same ParT with a global nuisance treatment. It gets mass decorrelation against all non-QCD classes via the adversarial method of Decorrelated jet substructure tagging using adversarial neural networks. It gets input-perturbation training with a single budget, following Improving Robustness of Jet Tagging Algorithms with Adversarial Training, which on its own flavor dataset cut the charm-vs-light degradation at perturbation size 0.05 from 21% to 8.2% while keeping nominal bottom-vs-light AUC at 0.946. No reported result on JetClass with global systematic training was found on Lacuna, so we will run it ourselves. We sweep the global penalty strength to trace a full frontier rather than pick one point.
The decisive ablation is the same pairwise-head model with the map replaced by all ones, $M_{ij,\nu}=1$ everywhere. This isolates scoping from architecture.
A second ablation applies the scoped penalties directly to the monolithic model's softmax-ratio scores, with no separate heads. This tests whether pairwise heads are needed at all.
A third baseline is one-versus-all decomposition with the same encoder and the same scoped penalties assigned per class.
For fairness, every model uses the same encoder size, training jets, nuisance-varied samples, number of epochs, and a matched hyperparameter search budget over the penalty strength $\lambda$. Beating the global baseline means dominating its frontier. At equal shift sensitivity on affected pairs, the scoped model must have higher nominal background rejection on unaffected pairs, by more than the seed-to-seed spread across five seeds, while losing no more than a few tenths of a point of clean ten-class accuracy.
Why not just…?
Why not just use one multi-class model with a global adversary? That is the strongest baseline above. The concern with it is specific: penalizing the mass dependence of the whole representation also erases the mass information that separates Z from Higgs to bottom quarks (about 91 vs 125 GeV). The frontier comparison measures exactly this cost. If the global model's frontier matches ours, the idea fails, and we will report that.
Why not just apply the scoped penalties to the softmax ratios of a monolithic model? This is the real threat. It keeps the cheapest architecture and keeps scoping. The softmax couples all classes through one normalization, so a penalty on the Z-vs-QCD ratio also moves logits used by other ratios. Whether that coupling actually hurts is an empirical question, not something we can argue in advance. The second ablation settles it. If it matches the pairwise-head model, the contribution reduces to the sensitivity map plus scoped loss, which is still useful but smaller, and the note's claim narrows accordingly.
Why not just measure per-pair efficiency scale factors in control samples, as experiments already do? Scale factors correct the average efficiency of a tagger, but their uncertainty is the residual systematic. A tagger whose scores move less under variations yields smaller scale-factor uncertainties. We therefore report, for each pair, the efficiency spread across nuisance variants at fixed working points. This is the quantity a scale-factor uncertainty would inherit.
Why not just wrap the global model in nuisance-aware prediction sets? Classification under Nuisance Parameters and Generalized Label Shift in Likelihood-Free Inference restores 90% coverage where standard conformal sets fall to 40–60%. It addresses coverage, not discrimination, and its cost grows quadratically in the number of classes. That approach is complementary, and the pairwise heads are a natural input to it. As a secondary check, we test whether sets built on scoped heads are smaller at equal coverage.
Why not just use one-versus-all? It needs fewer heads, but each head pools nine classes and faces imbalanced training data. A one-versus-all head for the Z boson cannot be mass-decorrelated against QCD without also losing mass information against the Higgs. The one-versus-all baseline tests this directly. We note that Chan et al. found one-versus-all more robust on their heavily imbalanced Tropic dataset, so this is not a foregone conclusion.
Proposed Experiments
The first decisive check is the sensitivity map itself. Before any constrained training, it must show that most of the 45 pairs shift little under any single nuisance.
The second is the frontier plot. For each pair, it shows nominal background rejection at 50% signal efficiency against shift sensitivity, where shift sensitivity is the change in rejection between nominal and Herwig samples or energy-scale-shifted samples. We compare scoped, all-ones, global, and monolithic-ratio models. The prediction is that only on map-zero pairs does the scoped model sit near the unconstrained model, while on map-one pairs it matches the global model.
The third is the mass-sculpting check on QCD jets. It measures the Jensen–Shannon divergence between the QCD mass spectrum before and after a tagger cut for each resonance-vs-QCD head. We confirm that Z-vs-Higgs-to-bottom separation keeps its mass dependence.
The fourth measures aggregation under shift: ten-class accuracy and calibration error on varied samples, comparing the coupling solve, the gated coupling, and majority voting. This tests whether irrelevant heads become more harmful off-nominal.
Finally, we repeat the scoped-versus-global comparison on the three-class flavor dataset of the adversarial jet-tagging paper, using its up/down variations. This checks the result against the 21%-versus-8.2% degradation and 0.946 AUC that paper reports, and tests whether the result transfers beyond JetClass.
References
- Interclass Interference Suppression in Multi-Class Problems
- Decorrelated jet substructure tagging using adversarial neural networks
- Improving Robustness of Jet Tagging Algorithms with Adversarial Training
- Systematics of quark/gluon tagging
- Particle Transformer for Jet Tagging
- Multi-Classification using One-versus-One Deep Learning Strategy with Joint Probability Estimates
- Uncertainty Quantification in Pairwise Difference Learning for Classification
- Classification under Nuisance Parameters and Generalized Label Shift in Likelihood-Free Inference