Splitting With Embeddings, Merging With Physics: Continual Discovery of New Boosted-Jet Classes

At the Large Hadron Collider, a heavy particle produced at high momentum decays into a narrow spray of hadrons that detectors record as a single wide "jet." Today's jet classifiers recognize a short, fixed list of known particles: top quarks, W, Z and Higgs bosons, and ordinary quark or gluon background. Anything new is either forced into one of those bins or thrown away as background. Anomaly detectors can flag that a jet looks unusual, but they stop there. They cannot say how many distinct kinds of unusual jets exist, and they cannot add those kinds to the classifier without breaking what it already knows. Machine learning has a toolkit for exactly this, called continual category discovery: clustering unlabeled data into new classes as it streams in, estimating how many classes there are, and avoiding forgetting of old ones. That toolkit has only been tested on natural images.

We argue that moving it to jets fails in a predictable, physics-specific way, and that the fix is simple. A learned jet embedding tends to place the same particle at different momenta in different regions of feature space, so a discovery algorithm counts one new particle as several. Hand-crafted substructure observables have the opposite weakness. They are boost-stable once computed on jets clustered with a momentum-dependent radius, but they are blind to differences such as quark flavour, so they merge distinct particles. Our proposal fuses the two streams asymmetrically. The deep embedding decides when a cluster should split, and the boost-normalized physics observables decide when two clusters should merge. The goal is to recover the right number of new classes and to assign jets to them correctly.

Core Claim

In continual discovery of jet classes, the dominant error is miscounting classes, and it has two opposite sources. Deep embeddings over-split along the momentum axis. Physics observables under-split along the flavour axis. A discovery procedure that proposes splits in embedding space but accepts merges only when the clusters agree on adaptively clustered substructure observables should estimate the class count more accurately and discover classes more accurately than either stream alone, or than both streams simply concatenated. Concretely, we predict three things. First, the class count estimated from embeddings alone will grow as the momentum range of the data stream widens. Second, our estimate will stay flat. Third, the gain will disappear if the physics stream is computed on fixed-radius jets instead of variable-radius ones.

Background and Positioning

The adaptive-clustering literature supplies the physics stream. Variable-radius clustering shrinks the jet's angular size as its transverse momentum $p_T$ grows, following the decay opening angle $\Delta R \approx 2m/p_T$, where $m$ is the particle mass. In searches for $H \to hh \to 4b$, Fat b-jet analyses using old and new clustering algorithms in new Higgs boson searches at the LHC shows that this prevents mass-peak shifts and raises significance from 10.799 (fixed radius) to 12.897. How to decontaminate overlapping fat jets instead stops clustering when the jet mass jumps, which keeps neighbouring decays out of a jet. Both papers motivate our view that boost-dependent jet boundaries are what make substructure comparable across momenta. Both also work in a closed world with a known target. We reuse their clustering rules as fixed preprocessing for one input stream. We do not learn the clustering itself.

The substructure observables we compute on these jets come from Identifying boosted objects with N-subjettiness, which measures how well a jet's energy lines up with N axes, and Energy correlation functions for jet substructure, which builds axis-free angular correlations. We also use the groomed quantities from soft-drop declustering studied in Jet Substructure Studies with CMS Open Data. That paper also shows that these observables are well modelled by simulation, which matters because they will anchor merge decisions.

The deep stream comes from self-supervised jet encoders. Large-Scale Pretraining and Finetuning for Efficient Jet Classification in Particle Physics pretrains a Transformer contrastively using symmetry and soft-smearing augmentations. RINO: Renormalization Group Invariance with No Labels builds views from the clustering history and reports 15–26% accuracy gains under domain shift. Neither paper discovers classes; both end in closed-world fine-tuning. We use one of these encoders frozen.

The discovery machinery comes from vision. PromptCCD: Learning Gaussian Mixture Prompt Pool for Continual Category Discovery models classes as Gaussian components over frozen features and estimates their number with a split-and-merge rule. Class-Incremental Novel Class Discovery replays features sampled from stored class Gaussians to avoid forgetting. Incremental Generalized Category Discovery anchors classes at density peaks. That paper also reports that discovery collapses when new classes arrive together with a domain shift: final discovery accuracy is 28.4% and forgetting 27.6% on iNatIGCD. This constrains our design. Collider data shifts across run conditions, so our merge criterion relies on observables known to be stable.

The fusion literature constrains how we combine streams. Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion finds that adding or concatenating hand-crafted features to deep embeddings beats cross-attention. We therefore use no learned fusion module. Our departure is to give each stream a different job instead of pooling them.

The closest jet work is Variational autoencoders for anomalous jet tagging. It detects multi-prong anomalies (AUC up to 0.913 with outlier exposure) but only as a binary outlier score. Our proposal starts where that work stops: it partitions the anomalies and grows the classifier.

Research Program

Each jet is described by two vectors. The first is a frozen embedding $e$ from a contrastively pretrained Transformer. To avoid leaking future classes, it is pretrained only on jets from the classes known at the first stage. The second is a physics vector $\phi$: groomed mass, $N$-subjettiness ratios $\tau_2/\tau_1$, $\tau_3/\tau_2$, $\tau_4/\tau_3$, energy-correlation ratios, and soft-drop $z_g$ and $\theta_g$. All of these are computed after reclustering the constituents with a variable radius $R_{\text{eff}} = \rho/p_T$, clipped to $[R_{\min}, R_{\max}]$. Because a new particle's mass is unknown, we compute $\phi$ for a small grid of $\rho$ values. Merge tests use the best-matching $\rho$ for each pair of clusters.

Discovery runs stage by stage over a stream that mixes known and unknown classes. In each stage, a Gaussian mixture is fitted over $e$. Old classes are kept alive by replaying samples from their stored Gaussians, following the FRoST approach, and components are initialized at density peaks. Splits are proposed with the Hastings-ratio criterion of PromptCCD operating on $e$ only.

The departure from PromptCCD is the merge step. Two components $j$ and $k$ are merged whenever their class-conditional distributions of the physics vector are statistically indistinguishable. We define the merge score as

$$s_{jk} = \min_{\rho}\; \mathrm{MMD}\big(P_j(\phi_\rho),\, P_k(\phi_\rho)\big),$$

where $P_j(\phi_\rho)$ is the distribution of physics observables of jets assigned to component $j$, computed at radius scale $\rho$, and MMD is the maximum mean discrepancy, a kernel two-sample distance. A merge is accepted if $s_{jk}$ falls below a threshold. The threshold is calibrated on stage-0 classes by artificially splitting each known class into $p_T$ halves and requiring those halves to merge, while known distinct classes do not.

The rule is deliberately one-sided. The physics stream can veto a split but never create one, because it cannot see flavour. A class such as Higgs to $b\bar b$ versus Z to quarks at similar mass is separated only by the embedding, and the physics stream merges only clusters that also look alike to it.

We will also analyse when this rule must fail. If a genuinely new particle shares mass, prong count and groomed substructure with a known one and differs only in flavour, the merge test will wrongly fuse them whenever their embeddings also overlap. We will measure how often this happens on the Higgs flavour modes.

Baselines

The naive reference is k-means on the frozen embedding at each stage. Its cluster count is chosen by silhouette score, and it has no replay; this shows what discovery looks like with no continual or physics machinery. We also run the same k-means with the true class count given. This oracle isolates assignment quality from counting errors. Beating the naive reference means higher continual clustering accuracy and lower class-count error.

The strongest existing discovery method is PromptCCD, ported to jets. Its Gaussian prompt pool and split-and-merge run over the same frozen encoder, with the prompt mechanism kept where the encoder allows it and a mixture head otherwise. The file reports 64.17% continual accuracy (cACC) for PromptCCD on CIFAR-100, versus 58.25% for PA-CGCD, in PromptCCD: Learning Gaussian Mixture Prompt Pool for Continual Category Discovery. We found no reported result on Lacuna for continual category discovery on any jet dataset, so we must run all numbers ourselves. We also port IGCD's density-peak nearest-neighbour classifier, which reports 28.4% final discovery accuracy versus 20.1% for Grow-and-Merge on iNatIGCD in Incremental Generalized Category Discovery.

All ported methods use the same encoder, the same stream, the same replay budget, and the same hyperparameter search budget. That budget is a fixed number of trials tuned on a stage-0 validation split that contains no novel classes. Beating PromptCCD means a lower absolute error in estimated class count at the final stage and higher final cACC, with forgetting no worse.

The anomaly-detection pipeline uses the outlier-exposed VAE of Variational autoencoders for anomalous jet tagging to flag jets. The flagged pool is then clustered offline with a Gaussian mixture over the same fused features, with the count chosen by the Bayesian information criterion.

As a ceiling, we train a fully supervised Particle Transformer on all labels. Particle Transformer for Jet Tagging reports 86.1% accuracy on JetClass. Discovery methods are not expected to reach this; it tells us how much label-free discovery costs.

Ablations isolate the key ingredient. We compare three variants: the merge test on fixed-radius physics observables instead of variable-radius ones; symmetric fusion, in which the concatenation $[e,\phi]$ feeds both split and merge decisions; and embedding-only and physics-only discovery.

Why not just…?

Why not just cluster the substructure observables directly? For mass and prong count this may work well, and it is a real threat on classes such as W versus top versus four-prong Higgs. Mass-related observables are exactly what Identifying boosted objects with N-subjettiness showed to separate prong counts. The physics-only baseline will reveal where it breaks. We expect failure on flavour-distinguished classes at similar mass and prong number, such as Higgs to $b\bar b$ versus Z to quarks. Within the Interaction networks for the identification of boosted H→bb decays work, separating Higgs to $b\bar b$ from background needed vertex and track information beyond kinematics. If physics-only discovery matches our method on the full stream, the embedding stream is unnecessary and we will say so.

Why not just concatenate both feature sets and run PromptCCD? This is the obvious use of the simple-fusion finding in Attention Is Not Always the Answer. The symmetric-fusion ablation is exactly this experiment. Our hypothesis is that in a concatenated space, the high-dimensional embedding still dominates distances, so momentum-driven over-splitting survives. If symmetric fusion matches asymmetric fusion on class-count error, the asymmetric rule adds nothing.

Why not just flag anomalies with a VAE and cluster them afterwards? This two-stage pipeline needs no continual machinery. But it has no mechanism to keep earlier discoveries stable as new data arrives, and its quality is capped by the anomaly score. That score loses power when mass-decorrelated: the VAE paper reports about 0.827 AUC for mass-decorrelated VAEs versus 0.913 with outlier exposure. The forgetting and per-stage accuracy curves of the VAE baseline will test this directly.

Why not just rescale jet angles by $p_T/m$ instead of variable-radius reclustering? This is a cheaper way to normalize boost and may capture most of the effect. We include it as a further ablation of the physics stream. If it matches variable-radius reclustering, we adopt it. Reclustering should matter mainly in crowded or contaminated jets, where rescaling a fixed cone does not remove foreign radiation, as How to decontaminate overlapping fat jets found.

Proposed Experiments

The first stream is built from JetClass, which has 100M Pythia8/Delphes jets in 10 classes. Stage 0 labels QCD, hadronic top and W. Later unlabeled stages introduce Z, then the Higgs modes including $H \to b\bar b$ and $H \to 4q$, mixed with ongoing known-class jets. JetClass jets come pre-clustered at $R=0.8$, so variable-radius reclustering here can only shrink jets. JetClass also covers a fairly narrow momentum window, so the boost-fracture effect should be weak.

For that reason, the decisive stream is a second one that we generate with the same MadGraph/Pythia8/Delphes chain used in the VAE study. It spans $p_T$ from roughly 200 to 2000 GeV, includes scalar resonances decaying to $b\bar b$ at several unknown masses, and includes $H \to hh \to 4b$ at the 2HDM benchmark point with $m_H \approx 700$ GeV. Event-level variable-radius clustering is possible there. To probe the domain-shift failure reported for IGCD, the final stage changes the pileup level.

Metrics are continual clustering accuracy with Hungarian matching, absolute class-count error, forgetting on old classes, and background rejection at fixed signal efficiency for each discovered class. We also measure mass sculpting as the Jensen–Shannon divergence between the QCD mass spectrum before and after selecting each discovered cluster. This checks that discovered "classes" are not just mass bins.

The decisive check plots estimated class count against the width of the momentum range in the generated stream. The prediction fails if embedding-only discovery does not over-count as the range widens, if our method does not stay near the true count, or if the fixed-radius ablation does not reintroduce over-counting. A second decisive check measures the false-merge rate on the Higgs flavour modes. This bounds the stated failure case of the physics-veto rule.