Deep Learning From Four Vectors

Pierre Baldi, Peter Sadowski, D. WhitesonarXiv 2022 · DOI · arXiv

TL;DR

This paper explores how deep learning (DL) has transformed particle physics by shifting from hand-crafted "expert features" to raw particle data (four-vectors). It demonstrates that deep neural networks can automatically discover complex physical laws, significantly outperforming traditional methods and even generalizing to search for hypothetical particles with unknown masses through "parameterized" architectures.

What Problem Does This Paper Address?

In high-energy physics (HEP), researchers look for rare signals (like the Higgs boson) buried under massive amounts of "background" noise. Traditionally, this was done using "multi-variate analysis" with shallow machine learning models like Boosted Decision Trees (BDTs).

The catch? Shallow models aren't very good at processing raw data. Physicists had to spend months "feature engineering"—manually calculating complex variables like invariant masses based on their domain knowledge—to help the models see the signal. This approach is labor-intensive and risks missing information that isn't captured by the chosen features. Furthermore, when searching for new particles, physicists often don't know the particle's mass, requiring them to train hundreds of separate models for every possible mass value.

Methods

The authors advocate for a shift toward Deep Learning from Four-Vectors. A "four-vector" is the fundamental way physicists describe a particle's state: its energy ($E$) and its momentum in three dimensions ($p_x, p_y, p_z$).

Key methodological contributions include:

  • Deep Neural Networks (DNNs): Using many-layered architectures to learn high-level abstractions directly from low-level four-vector inputs.
  • Parameterized Networks: Instead of training a separate model for every hypothetical mass, the authors include the mass ($\theta$) as an input feature. This allows a single network to learn the relationship between the mass parameter and the data distribution, enabling it to interpolate smoothly across a range of masses.
  • Physics-Aware Architectures: The paper discusses building physical symmetries directly into the network. For example, since the order of particles in a list shouldn't change the physics of an event, they use permutation invariance (via weight-sharing and pooling) to make the models more robust and efficient.
  • Hard vs. Soft Constraints: They describe methods to force networks to obey physical laws (like conservation of energy) either through penalty terms in the loss function (soft) or by designing the architecture itself to be mathematically incapable of violating the law (hard).

Datasets

The paper utilizes two primary benchmark cases:

  1. Higgs Benchmark: A simulated dataset designed to distinguish between a heavy Higgs signal and a top-quark background. It consists of 21 low-level features (momentum components of various particles).
  2. Exotic Particle Search ($X \to t\bar{t}$): Used to test parameterized networks, this dataset involves a hypothetical particle $X$ decaying into top quarks across a range of possible masses ($m_X$).

Key Insights and Ideas

  • The Death of Feature Engineering: The most striking "aha moment" is that DNNs trained on raw four-vectors achieve the same—or better—performance than shallow networks trained on "expert" features. This proves that deep networks are effectively "discovering" the physics (like invariant mass calculations) on their own.
  • Generalization via Parameterization: Parameterized networks don't just memorize specific mass points; they learn the underlying physics of the mass-data relationship. They can accurately predict results for mass values they were never shown during training.
  • Symmetry as a Prior: By incorporating symmetries like Lorentz invariance or permutation invariance, we provide the model with an "inductive bias." This reduces the amount of data needed for training and ensures the model doesn't learn physically impossible patterns.

Key Results

  • Performance Boost: In the Higgs benchmark, the DNN achieved an AUC of 0.880 using only low-level features, significantly outperforming shallow NNs (0.733) and BDTs (0.73).
  • Discovery Significance: Using deep learning increased the statistical significance of a potential discovery from $2.5\sigma$ to $4.9\sigma$ compared to shallow methods.
  • Seamless Interpolation: Parameterized networks matched the performance of "fixed" networks trained on specific masses and maintained that performance even when tested on intermediate mass values omitted from the training set.

Limitations and Future Work

While the paper highlights the power of DL, it acknowledges several challenges:

  • Interpretability: While we know DNNs are finding the "right" features, understanding exactly how they represent that physics internally remains an active area of research.
  • Systematic Uncertainties: In real-world physics, simulations aren't perfect. There is a risk that a deep network might learn to exploit "glitches" or artifacts in the simulation rather than real physics.
  • Variable Input Sizes: Standard DNNs expect a fixed number of inputs. However, particle collisions often produce a variable number of jets. The authors suggest that future work involving Recursive Neural Networks or Transformers (attention mechanisms) will be necessary to handle these "variable-size" sets of four-vectors more naturally.

Summary updated

Abstract

An early example of the ability of deep networks to improve the statistical power of data collected in particle physics experiments was the demonstration that such networks operating on lists of particle momenta (four-vectors) could outperform shallow networks using features engineered with domain knowledge. A benchmark case is described, with extensions to parameterized networks. A discussion of data handling and architecture is presented, as well as a description of how to incorporate physics knowledge into the network architecture.
BibTeX
@misc{baldi2022deep,
  author = {Pierre Baldi and Peter Sadowski and D. Whiteson},
  title = {{Deep Learning From Four Vectors}},
  year = {2022},
  eprint = {2203.03067},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2203.03067},
}
Figure