Higher Order Component Attribution via Learned Surrogates in High Energy Physics

Lauri Laatu, Aritra Bal, Benedikt Maier, Markus Klute, Michael SpannowskyarXiv 2026 · arXiv

This paper, presented at a NeurIPS 2026 workshop, introduces a novel interpretability framework for High Energy Physics (HEP) using Kolmogorov-Arnold Networks (KANs) as local surrogates. By extracting higher-order interaction terms from these surrogates, the authors demonstrate that state-of-the-art "physics-informed" models like the Particle Transformer (ParT) actually learn to rely on the same multi-particle correlations defined by theoretical physics, whereas standard transformer architectures do not.

TL;DR

The authors use KAN-based surrogates to "peek inside" black-box jet taggers, extracting 2nd and 3rd-order interaction coefficients. They find that the Particle Transformer (ParT) successfully aligns its internal logic with physical energy correlation functions, while standard Multi-Head Attention (MHA) fails to capture these fundamental physical structures despite achieving high accuracy.

What Problem Does This Paper Address?

In particle physics, "jets" are sprays of particles produced in high-energy collisions. Identifying the origin of these jets (e.g., a simple quark vs. a complex top quark decay) is a task called jet tagging.

While modern Deep Neural Networks (DNNs) like the Particle Transformer (ParT) have achieved state-of-the-art accuracy, they are often treated as black boxes. Physicists traditionally use hand-crafted observables called Energy Correlation Functions (ECFs) to describe jet substructure based on 2-body and 3-body particle interactions. The gap is clear: we know ParT works, but we don't know if it works because it has "learned the physics" of ECFs or if it is simply finding clever algorithmic shortcuts (clever "cheating").

Methods

The researchers propose a method for Higher-Order Component Attribution:

  1. Local Surrogates: They take a pre-trained "target" model (like ParT or a standard transformer) and probe it by applying small Gaussian perturbations to the input particle kinematics.
  2. KAN Training: They train a Kolmogorov-Arnold Network (KAN) to act as a local surrogate, mapping these input perturbations to the target model's output shifts. KANs are used because their b-spline activations allow for smooth, analytically differentiable surfaces.
  3. Interaction Extraction: Because the KAN is analytically differentiable, the authors compute the Hessian (2nd order) and 3rd-order derivative tensors. These represent the model's learned 2-body and 3-body interactions, respectively.
  4. Physical Alignment: They compare these extracted "learned" interactions against the "ground truth" physical contributions from ECFs ($e_2$ and $e_3$) using Spearman rank correlation ($R_{CORR}$).

Datasets

The study utilizes the JETCLASS dataset, a large-scale benchmark for HEP. They focus on three representative jet topologies:

  • 1-Prong: Light quarks or gluons ($q/g$).
  • 2-Prong: $Z$ boson decays ($Z \to q\bar{q}$).
  • 3-Prong: Top quark decays ($t \to bq\bar{q}$).
Representative jet constituents in the (η, ϕ) plane for one-, two-, and three-pronged topologies.

Key Insights and Ideas

  • Physics-Informed Bias Works: ParT includes explicit $\Delta R$ (angular separation) terms in its attention mechanism. The study shows this "bias" successfully nudges the model to align with physical $e_2$ and $e_3$ correlators.
  • MHA is a Black Box: Standard Multi-Head Attention (MHA) shows near-zero correlation with physical observables. It reaches high accuracy, but its internal "reasoning" does not mirror known physics.
  • The Depth Paradox: Increasing the number of transformer blocks (depth) improves classification accuracy but does not necessarily improve physical alignment. For MHA, adding depth actually decreased alignment with physical 2-body interactions.
  • Fidelity of KANs: The KAN surrogates achieved remarkably high fidelity ($R^2 \approx 0.99$ for ParT), proving they are excellent tools for capturing the local behavior of complex transformers.

Key Results

The primary quantitative finding is the difference in Rank Correlation ($R_{CORR}$) between the models and physical ECFs:

  • 2-Body Interactions: For 2-prong jets, ParT showed a clear positive correlation ($R_{CORR} \approx 0.3$), while standard MHA showed essentially none.
  • 3-Body Interactions: While weaker, ParT still showed a statistically significant positive correlation with 3-body physical interactions, whereas MHA did not.
  • Fidelity: As shown in Table 2 of the paper, the KAN surrogate captures 99% of the variance in ParT's output moves, compared to ~85% for an 8-block MHA model.
Figure 2: Ranked correlation coefficients for three representative types of jets with different radiation patterns (pronginess) show a positive correlation when using the ParT architecture for the pairwise scenario (left) for two and three-pronged jets. The triplet scenario (right) shows a weaker but still positive correlation for the same architecture. Each bar is the mean over the 50 jets of that class for the single 8-block network of each architecture with the highest test accuracy, and the error bars show ±1 standard deviation of the per-jet correlations.

Limitations and Future Work

  • Order Limit: The study only goes up to 3rd-order interactions (triplets). Many complex jet features may reside in even higher-order correlations.
  • Class Scope: The analysis was limited to a few standard jet classes. The authors suggest extending this to more "exotic" substructures (e.g., beyond the Standard Model physics).
  • Computational Cost: Training a KAN surrogate for every jet is computationally intensive, which limited the sample size for the correlation analysis to 50-1000 jets per class.

Summary updated

Abstract

Experimental High Energy Physics (HEP) has adopted the transformer architecture for identification of jets, particularly the state-of-the-art Particle Transformer (ParT) which injects explicit information about particle correlations into the attention mechanism. While able to achieve state-of-the-art results in accuracy, interpretability to its effectiveness has been limited. In this paper we propose component attribution methods using Kolmogorov-Arnold Networks (KANs) that directly tie back to the physical descriptors of jets, to probe whether mechanisms such as Multi-Head Attention (MHA), or the ParT architecture, are able to learn these as discriminating features.
BibTeX
@misc{laatu2026higher,
  author = {Lauri Laatu and Aritra Bal and Benedikt Maier and Markus Klute and Michael Spannowsky},
  title = {{Higher Order Component Attribution via Learned Surrogates in High Energy Physics}},
  year = {2026},
  eprint = {2611.08656},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08656},
}
Figure