Abstract
We apply adversarial domain adaptation in unsupervised setting to reduce sample bias in a supervised high energy physics events classifier training. We make use of a neural network containing event and domain classifier with a gradient reversal layer to simultaneously enable signal versus background events classification on the one hand, while on the other hand minimizing the difference in response of the network to background samples originating from different Monte Carlo models via adversarial domain classification loss. We show the successful bias removal on the example of simulated events at the Large Hadron Collider with signal versus background classification and discuss implications and limitations of the method.
Introduction
In High Energy Physics (HEP), identifying rare signals like the Higgs boson ($t\bar{t}H$) amidst massive backgrounds (such as $t\bar{t} + \text{jets}$) is a fundamental challenge. Because real-world data lacks ground-truth labels, researchers rely on Monte Carlo (MC) simulations for training machine learning classifiers. However, different simulators—such as MadGraph and Powheg—rely on varying physical assumptions, leading to divergent data distributions. This discrepancy, known as "sample bias," introduces significant systematic uncertainties that can hinder the discovery of new physics. This paper explores the use of Domain Adversarial Neural Networks (DANN) to mitigate this bias by learning domain-invariant features.
Methodology
The core of the proposed approach is the Domain Adversarial Neural Network (DANN) architecture. As illustrated in Figure 1, the network is composed of three primary components:
- Feature Extractor ($G_f$): Maps the input data (41 kinematic variables) into a high-dimensional feature space.
- Label Predictor ($G_y$): Classifies events as either Signal or Background based on the extracted features.
- Domain Classifier ($G_d$): Attempts to identify which simulator (Source or Target) produced the input data.
The critical innovation is the Gradient Reversal Layer (GRL) placed between the feature extractor and the domain classifier. During forward propagation, the GRL acts as an identity transform. However, during backpropagation, it multiplies the gradients by a negative factor ($-\lambda$). This adversarial setup forces the feature extractor to evolve such that the domain classifier cannot distinguish between the source and target domains, effectively stripping the features of simulator-specific quirks while retaining their predictive power for physics classification.
Experimental Setup
The study utilizes simulated Large Hadron Collider (LHC) events at $\sqrt{s} = 13$ TeV. The signal consists of $t\bar{t}H$ events ($H \to b\bar{b}$), while the background is $t\bar{t} + b\bar{b}$. The "Source" domain was generated using MadGraph matched to Herwig6/Pythia6, and the "Target" domain used PowhegBox + OpenLoops matched to Pythia8. The input features include 41 geometrical and kinematic quantities, such as angular distances and masses of jet systems.
Two loss configurations were tested: Set-up A (standard softmax/cross-entropy) and Set-up B (a novel linear loss for the domain classifier). The researchers found that Set-up B, combined with the RMSProp optimizer, provided significantly better training stability compared to the standard ADAM optimizer.
Key Results and Discussion
The application of DANN led to a dramatic reduction in sample bias. Without adaptation, the classifier's response varied significantly between simulators, leading to a relative uncertainty in signal purity of approximately 47%. With the adversarial training (Set-up B), this uncertainty dropped to just 0.3%. Furthermore, the Area Under the ROC Curve (AUC) on the target domain improved from 0.657 to 0.760, demonstrating that the model became substantially more robust to unseen simulation styles.
To tune the hyperparameter $\lambda$ without target labels, the authors utilized the Kolmogorov-Smirnov (KS) distance to measure domain agreement. This allows the model to be optimized using unlabeled data, which is essential for applying the method to real experimental collision data.
Conclusion and Future Work
Adversarial domain adaptation effectively bridges the gap between different simulation models in HEP. While highly successful, the method faces challenges such as "label shift"—where the signal-to-background ratio differs between domains—which can reintroduce minor biases. Future research will focus on applying this technique directly to unlabeled real-world collision data and extending the architecture to handle multiple simulation domains simultaneously to create a truly universal physics feature extractor.
Summary updated