Learning Broken Symmetries with Approximate Invariance

Seth Nabat, Aishik Ghosh, Edmund Witkowski, Gregor Kasieczka, Daniel WhitesonarXiv 2024 · DOI · arXiv

Introduction

In fields such as high-energy physics and computer vision, symmetries—like rotation or translation invariance—serve as powerful inductive biases. Incorporating these symmetries into neural network architectures allows models to learn more efficiently from limited data. However, real-world data often exhibits "broken" symmetries due to experimental noise or detector artifacts. Standard physics-informed models that enforce perfect symmetry face a performance ceiling when confronted with these subtle asymmetries. Conversely, unconstrained general-purpose models require vast amounts of data to learn the underlying patterns. This paper proposes a hybrid "dual-subnet" architecture that bridges this gap, combining the data efficiency of symmetry-constrained models with the high-end performance of flexible networks.

Methodology: The Hybrid Dual-Subnet Architecture

The proposed model balances a symmetry-constrained "Invariant Subnet" with a flexible "Agnostic Subnet." The core innovation lies in a learned weighting mechanism that determines the relative contribution of each branch based on the input context.

The hybrid model architecture combines an Invariant Subnet (MLP) and an Agnostic Subnet (PFN) via a context-aware weighted average.

The architecture consists of three primary components:

  1. Invariant Subnet: An MLP that processes only invariant features (e.g., Minkowski inner products), ensuring its output remains unchanged under specific transformations.
  2. Agnostic Subnet: A Particle Flow Network (PFN) that operates on raw data (four-vectors) without built-in symmetry constraints.
  3. Learned Symmetry Factor: A separate small network that evaluates the context (such as transverse momentum, $p_T$) to produce a weight $w \in [0, 1]$.

The final representation is calculated as a weighted average: $x = w \cdot (\text{Agnostic}) + (1 - w) \cdot (\text{Invariant})$. This allows the model to rely on symmetry where it holds and transition to the unconstrained model where the symmetry is broken.

Experimental Setup

The approach was evaluated on a simulated high-energy physics task: distinguishing between a $Z$ boson decaying into two muons (signal) and a background process ($pp \to \gamma \to \mu^+\mu^-$). To simulate realistic symmetry breaking, the authors introduced a $p_T$-dependent resolution degradation in the muon momentum. This effect makes the measured mass dependent on $p_T$, violating theoretical Lorentz invariance. Data was generated using MadGraph5 at $\sqrt{s} = 13$ TeV.

Results and Discussion

The hybrid model demonstrates superior performance across varying dataset sizes, effectively capturing the strengths of both constituent subnets.

AUC performance vs. training set size. The hybrid model (red) matches the early efficiency of the invariant classifier (blue) and the asymptotic performance of the general classifier (black).

Data Efficiency and Asymptotic Performance

As shown in the performance metrics, the hybrid model significantly outperforms the general Agnostic network when training data is scarce, matching the rapid convergence of the Invariant network. As the dataset size increases, the hybrid model "escapes" the performance ceiling of the purely invariant approach, eventually matching or exceeding the high-end performance of the unconstrained network.

Interpretability of Symmetry Breaking

A key insight is that the model autonomously learns where symmetry breaks. By analyzing the learned weight $w$, the researchers found that the model shifts its reliance toward the Agnostic subnet at higher $p_T$ values, precisely where detector effects most severely disrupt Lorentz invariance. This provides a physical diagnostic tool for identifying regions of experimental data where theoretical assumptions may falter.

Conclusion

This research introduces a flexible framework for incorporating approximate symmetries into machine learning models. By treating symmetry as a starting point rather than a rigid constraint, the hybrid architecture achieves both high data efficiency and superior final accuracy. Future work may explore automated methods for identifying the context variables that drive symmetry breaking and applying this "dual-subnet" approach to broader domains like image processing and experimental hardware calibration.

Summary updated

Abstract

Recognizing symmetries in data allows for significant boosts in neural network training, which is especially important where training data are limited. In many cases, however, the exact underlying symmetry is present only in an idealized dataset, and is broken in actual data, due to asymmetries in the detector, or varying response resolution as a function of particle momentum. Standard approaches, such as data augmentation or equivariant networks fail to represent the nature of the full, broken symmetry, effectively overconstraining the response of the neural network. We propose a learning model which balances the generality and asymptotic performance of unconstrained networks with the rapid learning of constrained networks. This is achieved through a dual-subnet structure, where one network is constrained by the symmetry and the other is not, along with a learned symmetry factor. In a simplified toy example that demonstrates violation of Lorentz invariance, our model learns as rapidly as symmetry-constrained networks but escapes its performance limitations.
BibTeX
@misc{nabat2024learning,
  author = {Seth Nabat and Aishik Ghosh and Edmund Witkowski and Gregor Kasieczka and Daniel Whiteson},
  title = {{Learning Broken Symmetries with Approximate Invariance}},
  year = {2024},
  eprint = {2412.18773},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.18773},
}
Figure