Deep extreme feature extraction: New MVA method for searching particles in high energy physics

Chao Ma, Jinhui Xu, Tiancheng Hou, Bin Lan, Zhang ZhenhuaFilomat 2018 · DOI

TL;DR

This paper introduces Deep Extreme Feature Extraction (DEFE), a novel ensemble deep learning method for identifying Higgs boson decays in high-energy physics. DEFE achieves state-of-the-art performance by training a diverse set of neural feature learners guided by an implicit controller, leading to a significant reduction in error rate and a higher discovery significance compared to traditional Deep Neural Networks.

What Problem Does This Paper Address?

High-energy physics experiments, like those at the Large Hadron Collider (LHC), generate massive amounts of data from particle collisions. Identifying rare signal events (e.g., Higgs boson decays) amidst a sea of background noise is a critical challenge. Traditional machine learning methods, including Deep Neural Networks (DNNs), have been employed, but they often struggle to fully extract subtle, discriminative features, especially when dealing with complex, high-dimensional datasets. Specifically, while DNNs can learn features automatically, they may not always capture all the important patterns, particularly those that are not immediately obvious from low-level detector measurements. This paper aims to improve the accuracy and sensitivity of particle identification by developing a more effective feature extraction technique.

Methods

The core innovation is the Deep Extreme Feature Extraction (DEFE) method. DEFE is an ensemble learning scheme that trains a diverse set of neural feature learners. Unlike standard ensemble methods that might average outputs or explicitly enforce diversity, DEFE uses an implicit neural controller that guides the gradient flows during training. This controller, not directly involved in the forward pass, ensures that each feature learner is trained to be distinct.

The process involves:

  1. Discriminative Partition: The input data space is divided into several regions based on a weak classifier (the controller).
  2. Feature Extraction: For each partitioned region, a separate neural network (specifically, Stacked Denoising Autoencoders - SDAENNs) is trained to extract "extreme features."
  3. Ensemble Combination: The features extracted from all these learners are then fed into a final deep neural network (the predictor) to approximate the "extreme selection region" and make the final classification.

This approach allows for a "deep" post-processing of features, where the final network learns to select and abstract from the ensemble of learned features, avoiding the limitations of simple averaging.

Datasets

The experiment uses simulated data from the Higgs Boson Machine Learning Challenge. This dataset is generated by the ATLAS detector simulator and includes Higgs boson decays into τ+τ− events, mixed with various backgrounds, notably Z bosons decaying into τ+τ−. The dataset consists of 250,000 events, each described by 30 input attributes (17 low-level detector features and 13 high-level features calculated by physicists).

Key Insights and Ideas

  • Implicit Controller for Diversity: The key to DEFE's success lies in its implicit neural controller. Instead of explicit diversity-promoting mechanisms, this controller subtly guides gradients, naturally encouraging feature learners to specialize and capture different aspects of the data.
  • "Extreme" Features: By partitioning the data and training specialized feature extractors on each partition, DEFE aims to capture "extreme" or highly discriminative features that might be missed by a single, monolithic network.
  • Deep Ensemble Learning: DEFE is described as a "deep" ensemble because it allows for a sophisticated post-processing of the learned features, rather than just a simple combination of base model outputs. This enables the model to learn how to best utilize the diverse features.
  • Uncovering Hidden Patterns: The paper highlights that DEFE can identify important patterns in high-level features that are missed by standard DNNs, suggesting it can extract more nuanced information from the data.

Key Results

  • Improved Accuracy: DEFE achieved an AUC (Area Under the ROC Curve) of 0.916, surpassing other models.
  • Reduced Error Rate: Compared to a classic Deep Neural Network (DNN), DEFE reduced the error rate by approximately 37%.
  • High Accuracy: The accuracy broke through 90% for the first time in this context.
  • Significant Discovery: The discovery significance reached 6.0σ, a crucial threshold in particle physics for claiming a discovery.
  • Benefit from High-Level Features: Unlike standard DNNs, DEFE significantly benefits from the inclusion of high-level features, indicating its superior ability to leverage them.

Limitations and Future Work

  • Training Complexity: The paper notes that training the gating function (the implicit controller) can be challenging as it's not directly part of the main loss function or network structure. They propose a linear combination with the main loss to address this.
  • Sequential Training: The current DEFE algorithm trains components sequentially and greedily, which might not be the most optimal or efficient approach.
  • Future Work: The authors suggest that proposing further training algorithms to train an EFE model universally and efficiently remains an open question. They also aim to improve the analyzing quality of HEP data and the statistical significance of confirming physical facts.

Summary updated

Abstract

In this paper, we propose Deep Extreme Feature Extraction (DEFE), a new ensemble MVA method for searching ?+?- channel of Higgs bosons in high energy physics. DEFE can be viewed as a deep ensemble learning scheme that trains a strongly diverse set of neural feature learners without explicitly encouraging diversity and penalizing correlations, which is achieved by adopting an implicit neural controller (not involved in feed forward computation) that directly controls and distributes gradient flows from higher level deep prediction network. Such model-independent controller results in that every single local feature learned are used in the feature-to-output mapping stage, avoiding the blind averaging of features. DEFE makes the ensembles ?deep? in the sense that it allows deep post-process of these features that try to learn to select and abstract the ensemble of neural feature learners. Based the construction and approximation of the so-called extreme selection region, the DEFE model is able to be trained efficiently, and extract discriminative features from multiple angles and dimensions, hence the improvement of the selection region of searching new particles in HEP can be achieved. With the application of this model, a selection region full of signal processes can be obtained through the training of miniature collision events set. In comparison with the Classic Deep Neural Network, DEFE shows a state-of-the-art performance: the error rate has decreased by about 37%, the accuracy has broken through 90% for the first time, along with the discovery significance has reached a standard deviation of 6.0?. Experimental data shows that DEFE is able to train an ensemble of discriminative feature learners that boosts the over performance of final prediction. Furthermore, among high-level features, there are still some important patterns that are unidentified by DNN and are independent of low-level features, while DEFE is able to identify these significant patterns more efficiently
BibTeX
@inproceedings{ma2018deep,
  author = {Chao Ma and Jinhui Xu and Tiancheng Hou and Bin Lan and Zhang Zhenhua},
  title = {{Deep extreme feature extraction: New MVA method for searching particles in high energy physics}},
  booktitle = {Filomat},
  year = {2018},
  doi = {10.2298/fil1805711m},
}
Figure