Semi-visible jets, energy-based models, and self-supervision

Luigi Favaro, Michael Krämer, Tanmoy Modak, Tilman Plehn, Jan Rüschkamp

TL;DR

This paper introduces DarkCLR, a novel self-supervised learning framework for detecting "semi-visible jets" at the LHC. By learning representations that are invariant to physical transformations and incorporating "anomalous" augmentations, DarkCLR achieves robust and sensitive detection of these elusive signals, outperforming traditional supervised methods.

What Problem Does This Paper Address?

The search for new physics at the Large Hadron Collider (LHC) often relies on identifying anomalies in the vast amounts of data. Model-agnostic anomaly detection methods are crucial as they don't require prior assumptions about the nature of new physics. One such signature of new physics could be "semi-visible jets," which arise from the decay of hypothetical dark sector particles. These jets are challenging to detect because they have fewer visible particles than standard jets and their properties can vary significantly. Traditional anomaly detection methods often rely on density estimation or classification in semi-supervised settings, but these can be sensitive to coordinate transformations and require some knowledge of the signal. This paper aims to develop a robust, model-agnostic anomaly detection framework for semi-visible jets that is invariant to physical transformations and doesn't require explicit signal labels during training.

Methods

The core of the proposed framework is DarkCLR, a self-supervised contrastive learning representation (CLR) method. Here's a breakdown of the key techniques:

  • Self-Supervised Contrastive Learning (CLR): Instead of relying on labeled data, DarkCLR learns representations by comparing an original jet with augmented versions of itself. It uses a transformer encoder network to process jet constituents and create permutation-invariant representations.
  • Physical Augmentations: To ensure the learned representations are invariant to physical transformations, DarkCLR applies rotations and translations to the jet constituents. These are considered "positive" augmentations.
  • Anomalous Augmentations: To specifically train the model to distinguish semi-visible jets, DarkCLR introduces "anomalous" augmentations. For semi-visible jets, this involves randomly dropping constituents from a jet with a certain probability and rescaling the jet's momentum. This mimics a key characteristic of semi-visible jets (fewer constituents) without requiring explicit signal knowledge.
  • CLR-inspired Anomaly Score ($S_{CLR}$): After training the CLR model on background data, an anomaly score is derived from the norm of the learned representation vector. A regularization term is added to penalize the norm of background representations, making outliers (potential signals) have larger norms.
  • Normalized Autoencoder (NAE): As a second anomaly score ($S_{NAE}$), the paper uses a normalized autoencoder trained on the representations learned by DarkCLR. The NAE acts as a density estimator, and its reconstruction error serves as an anomaly score. This approach is more robust to the high dimensionality of the representations.

Datasets

The paper uses simulated datasets for training and evaluation:

  • QCD Background: Simulated di-jet events generated from leading-order QCD processes.
  • Semi-visible Jet Signals: Simulated semi-visible jets generated using the Pythia Hidden Valley module. These are based on a benchmark scenario with a $Z'$ mediator and dark quarks, with variations in parameters like the invisible fraction ($r_{inv}$) and dark meson mass scale ($m_{\pi/\rho}$). The simulation pipeline involves Madgraph5 for the hard process, Pythia 8 for showering and hadronization, and Delphes 3 for detector simulation. Jets are reconstructed using the anti-$k_t$ algorithm with $R=0.8$.

Key Insights and Ideas

  • Anomalous Augmentations are Key: Introducing "anomalous" augmentations that mimic signal characteristics (like dropping constituents for semi-visible jets) is crucial for training a model that can effectively distinguish signals from background, even without explicit signal labels.
  • Transformer for Jet Representations: The transformer architecture proves effective at learning permutation-invariant representations of jets, capturing complex correlations between constituents.
  • Robustness to Model Variations: DarkCLR demonstrates remarkable robustness to variations in the phenomenological parameters of the semi-visible jet signal (e.g., invisible fraction, meson masses), outperforming supervised classifiers that are sensitive to these variations.
  • Self-Supervision for Model Agnosticism: The self-supervised nature of DarkCLR, trained solely on background data, makes it inherently model-agnostic, a significant advantage for broad searches for new physics.
  • Complementary Anomaly Scores: Combining a CLR-derived score (based on representation norm) with a density-based score from a NAE trained on these representations provides a powerful anomaly detection system.

Key Results

  • Improved Background Rejection: DarkCLR achieves significantly better background rejection ($ \epsilon_B^{-1} $) at low signal efficiencies ($ \epsilon_S $) compared to previous state-of-the-art methods like NAEs trained on jet images, Dirichlet variational autoencoders, and invertible neural networks. For example, at $ \epsilon_S = 0.2 $, DarkCLR achieves $ \epsilon_B^{-1} = 59(1) $, compared to $ 41(1) $ for NAE on jet images.
  • Robustness Demonstrated: A supervised classifier trained on a specific semi-visible jet signal shows a large drop in performance when tested on datasets with varied signal parameters. In contrast, DarkCLR maintains strong performance across these variations.
  • Effective Representation Learning: The learned representations from DarkCLR are shown to be informative, outperforming constituent-level information for a simple linear classifier test.
  • Sensitivity to Embedding Dimension: DarkCLR shows improved detection even for smaller embedding dimensions, while augmentation's impact is more pronounced for larger embedding spaces.

Limitations and Future Work

  • Assumed Uniform Dropping: The current anomalous augmentation assumes uncorrelated visible constituents are dropped uniformly. The authors suggest a dedicated study for other dropping mechanisms.
  • Detector Smearing and Non-uniform Decay: The framework is flexible enough to incorporate other transformations like detector smearing or non-uniform decay of dark sector particles.
  • Wider Signal Coverage: Future work could include additional augmentations to cover a wider range of signal classes, especially those with higher jet multiplicity.
  • Interpretability and Dimensionality: Further studies are planned to investigate the effect of representation space dimensionality and the interpretability of the latent space.
  • Real Data Application: The authors foresee applying DarkCLR directly to real data to overcome potential biases from specific simulation choices, such as hadronization models.

Summary updated

Abstract

We present DarkCLR, a novel framework for detecting semi-visible jets at the LHC. DarkCLR uses a self-supervised contrastive-learning approach to create observables that are approximately invariant under relevant transformations. We use background-enhanced data to create a sensitive representation and evaluate the representations using a CLR-inspired anomaly score and a normalized autoencoder as density estimators. Our results show a remarkable sensitivity for a wide range of semi-visible jets and are more robust than a supervised classifier trained on a specific signal.
BibTeX
@inproceedings{favaro2025semi,
  author = {Luigi Favaro and Michael Krämer and Tanmoy Modak and Tilman Plehn and Jan Rüschkamp},
  title = {{Semi-visible jets, energy-based models, and self-supervision}},
  booktitle = {SciPost},
  year = {2025},
  doi = {10.21468/scipostphys.18.2.042},
}
Figure