Safety of Quark/Gluon Jet Classification

Alexis Romero, D. Whiteson, M. J. Fenton, Julian Collado, Pierre BaldiarXiv 2021 · DOI · arXiv

Abstract

The classification of jets as quark- versus gluon-initiated is an important yet challenging task in the analysis of data from high-energy particle collisions and in the search for physics beyond the Standard Model. The recent integration of deep neural networks operating on low-level detector information has resulted in significant improvements in the classification power of quark/gluon jet tagging models. However, the improved power of such models trained on simulated samples has come at the cost of reduced interpretability, raising concerns about their reliability. We elucidate the physics behind quark/gluon jet classification decisions by comparing the performance of networks with and without constraints of infrared and collinear safety, and identify the nature of the unsafe information by revealing the energy and angular dependence of the learned models. This in turn allows us to approximate the performance of the low-level networks (by 99% or higher) using equivalent sets of interpretable high-level observables, which can be used to probe the fidelity of the simulated samples and define systematic uncertainties.

Introduction

Distinguishing between jets initiated by quarks and gluons is a fundamental task at the Large Hadron Collider (LHC). While deep learning has significantly enhanced classification accuracy, it has introduced a "black box" problem. Modern Particle-Flow Networks (PFNs), which utilize low-level particle data, consistently outperform Energy-Flow Networks (EFNs), which are mathematically constrained to be Infrared and Collinear (IRC) safe. In Quantum Chromodynamics (QCD), IRC safety is the gold standard for theoretical reliability, as it ensures observables are insensitive to soft radiation or collinear splitting. The performance gap between PFNs and EFNs suggests that deep learning models exploit "unsafe" information that may not be accurately modeled in simulations, potentially leading to unreliable results when applied to real data.

Methodology

To investigate this performance gap, we employed a comparative strategy using simulated proton-proton collisions at $\sqrt{s}=14$ TeV. Events were generated with MadGraph5, showered with Pythia8, and passed through a Delphes simulation of an ATLAS-like detector. We analyzed three jet transverse momentum ($p_T$) ranges: 200–220 GeV, 500–550 GeV, and 1000–1100 GeV.

Our approach involved:

  1. Benchmarking: Using PFNs as the upper limit of unconstrained performance.
  2. IRC-Safe Baseline: Using EFNs to represent optimal IRC-safe performance.
  3. Probing Unsafety: Introducing EFN[κ], where the energy weighting exponent $\kappa$ is varied. While $\kappa=1$ corresponds to IRC safety, varying $\kappa$ allows us to identify which energy scales contribute most to classification.
  4. Observable Mapping: Attempting to replicate PFN performance using Linear Discriminant Analysis (LDA) on sets of interpretable high-level variables, including N-subjettiness, tower multiplicity ($n_t$), and Energy-Flow Polynomials (EFPs).

Probing the Performance Gap

A key finding is that the performance gap between PFNs and EFNs vanishes at the parton level, indicating that the "unsafe" information originates during the non-perturbative hadronization process. By testing different $\kappa$ values, we observed that the highest classification power occurs at small $\kappa$ (between $1/4$ and $1/2$), as shown in Figure 1. This suggests the networks gain an advantage by exploiting soft radiation.

AUC performance as a function of the energy weighting exponent $\kappa$ across different jet $p_T$ ranges. The peak at $\kappa < 1$ indicates that the most discriminative information is IRC-unsafe.

Much of this unsafe information is captured by simple tower multiplicity ($n_t$)—the count of calorimeter hits. While $n_t$ is theoretically unsafe because a single collinear split changes the count, it provides significant empirical discrimination.

Reclaiming Interpretability

The "black box" nature of PFNs can be largely demystified. By combining IRC-safe EFPs with a few specific IRC-unsafe variables (such as $n_t$ and unsafe EFPs), we can match PFN performance almost perfectly. As shown in Figure 2, an LDA model using these high-level observables achieves approximately 99% of the PFN's AUC across all energy ranges. For instance, in the 1000–1100 GeV range, the PFN achieved an AUC of 0.848, while our interpretable LDA model reached 0.844.

Comparison of PFN performance with LDA models using high-level observables. The LDA models, incorporating both safe and unsafe information, closely approximate the performance of the unconstrained PFN.

Conclusion

This work demonstrates that the performance gains of deep learning in jet tagging are not due to "hidden" physics but rather to specific, identifiable IRC-unsafe information—primarily soft, narrow-angle radiation from hadronization. By mapping this information to interpretable observables, we provide a framework for validating simulations and defining systematic uncertainties. This "safe" machine learning strategy allows physicists to reach the performance ceiling of black-box models while maintaining the transparency required for robust scientific discovery.

Summary updated

Abstract

The classification of jets as quark- versus gluon-initiated is an important yet challenging task in the analysis of data from high-energy particle collisions and in the search for physics beyond the Standard Model. The recent integration of deep neural networks operating on low-level detector information has resulted in significant improvements in the classification power of quark/gluon jet tagging models. However, the improved power of such models trained on simulated samples has come at the cost of reduced interpretability, raising concerns about their reliability. We elucidate the physics behind quark/gluon jet classification decisions by comparing the performance of networks with and without constraints of infrared and collinear safety, and identify the nature of the unsafe information by revealing the energy and angular dependence of the learned models. This in turn allows us to approximate the performance of the low-level networks (by 99\% or higher) using equivalent sets of interpretable high-level observables, which can be used to probe the fidelity of the simulated samples and define systematic uncertainties.
BibTeX
@misc{romero2021safety,
  author = {Alexis Romero and D. Whiteson and M. J. Fenton and Julian Collado and Pierre Baldi},
  title = {{Safety of Quark/Gluon Jet Classification}},
  year = {2021},
  eprint = {2103.09103},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.09103},
}

Loading related content...

Figure