Fast multi-geometry calorimeter simulation with conditional self-attention variational autoencoders

Dylan Smith, A. Ghosh, Junze Liu, Pierre Baldi, D. WhitesonarXiv 2024 · DOI · arXiv

TL;DR

Researchers have introduced SAVAE (Self-Attention Variational Autoencoder), a deep learning model that simulates particle showers in physics detectors across multiple different geometries. By leveraging self-attention and a clever geometry-conditioning technique, SAVAE is over 250 times faster and significantly more accurate than previous "geometry-aware" models, allowing physicists to simulate new detector designs without the massive computational cost of traditional methods.

What Problem Does This Paper Address?

In particle physics, simulating how a detector responds to a particle (like a photon hitting a calorimeter) is essential for data analysis. The gold standard, Geant4, uses Monte Carlo methods to model every microscopic interaction, but it is notoriously slow and computationally expensive.

Machine Learning (ML) models like GANs and VAEs can generate these simulations in milliseconds. However, most ML models are "fixed-geometry"—they are trained for one specific sensor layout. If a physicist wants to test a new detector design with different cell sizes, they usually have to generate a whole new dataset and train a new model from scratch.

Existing "geometry-aware" models that try to solve this are either too complex (requiring the coordinates of every single cell) or too slow (generating one cell at a time, like a chatbot generates words). This paper aims to find the "sweet spot": a model that is fast, accurate, and can handle various regular rectangular geometries.

Methods

The authors propose the Self-Attention Variational Autoencoder (SAVAE). Its architecture consists of:

  • Variational Autoencoder (VAE): A standard generative framework that maps input data to a compressed "latent space" and then reconstructs it.
  • Self-Attention Layers: Unlike standard convolutional layers that only look at nearby pixels, self-attention allows the model to weigh the importance of every cell in the detector relative to every other cell. This helps capture the global shape of a particle shower and handles variable-length data (different numbers of cells) gracefully.
  • Geometry Conditioning: To make the model "aware" of the layout, the authors concatenate a conditioning vector to the latent space. This vector $[\eta_-, \eta_+, \phi_-, \phi_+]$ simply tells the model how many cells are in each region of the detector.
  • Regularity Exploitation: Instead of specifying every cell's position, the model takes advantage of the fact that most detectors are built in regular segments, drastically reducing the complexity of the input.

Datasets

The study uses a high-granularity dataset based on the CaloGAN framework:

  • Source: Simulated calorimeter responses to 10,000 photons at a fixed energy of 65 GeV.
  • Granularity: The data was generated with 16 times more segmentation than standard datasets to allow for "re-binning" into various test geometries.
  • Geometries:
    • Training: Four configurations, including a non-uniform "transition region" geometry.
  • Unseen (Interpolation): Two geometries the model never saw during training, used to test if the model truly learned the relationship between geometry and physics.

Key Insights and Ideas

  • The Generality Spectrum: The authors visualize a spectrum from "Most Specific" (one model per layout) to "Most General" (point clouds). SAVAE occupies a middle ground that balances high performance with the flexibility needed for real-world detector regularity.
  • Parallelism is King: Previous geometry-aware models like GAAM were "autoregressive," meaning they predicted cells one by one. By using self-attention, SAVAE processes the entire detector image in parallel, leading to a massive leap in generation speed.
  • Physics-Preserving Interpolation: The model doesn't just memorize layouts; it successfully interpolates. It can accurately simulate a $(24 \times 24)$ grid even if it was only trained on $(12 \times 12)$ and $(48 \times 48)$ grids.

Key Results

  • Accuracy: SAVAE outperformed the previous state-of-the-art (GAAM) by over 70% across key physics metrics. In terms of the Wasserstein distance (a measure of how close two distributions are), SAVAE was sometimes two orders of magnitude more accurate.
  • Speed: SAVAE is blazingly fast. It takes 0.568 ms to generate a shower, compared to 140.5 ms for GAAM. This makes it viable for the high-throughput demands of modern physics experiments.
  • Parity with Specialists: Surprisingly, the "general" SAVAE model performed nearly as well as models trained specifically for a single geometry, proving that you don't have to sacrifice much accuracy for flexibility.

Limitations and Future Work

  • Dimensionality: The current work focuses on 2D images representing layers of a calorimeter. Real showers are 3D, and future work needs to scale this to model multiple longitudinal layers simultaneously.
  • Geometry Complexity: While the model handles regular rectangular grids and transition regions well, it hasn't yet been tested on more exotic, non-rectangular (e.g., hexagonal) geometries.
  • Energy Scaling: The current study used fixed-energy photons (65 GeV). A truly "foundational" model would need to be conditioned on incident particle energy as well.

Summary updated

Abstract

The simulation of detector response is a vital aspect of data analysis in particle physics, but current Monte Carlo methods are computationally expensive. Machine learning methods, which learn a mapping from incident particle to detector response, are much faster but require a model for every detector element with unique geometry. Complex geometries may require many models, each with their own training samples and hyperparameter tuning tasks. A promising approach is the use of geometry-aware models, which condition the response on the geometry, but current efforts typically require cumbersome full geometry specification. We present a geometry-aware model that takes advantage of the regularity of detector segments, requiring only the definition of cell sizes across regular segments. This model outperforms the current state of the art by over 70% across several key metrics including the Wasserstein distance metric.
BibTeX
@misc{smith2024fast,
  author = {Dylan Smith and A. Ghosh and Junze Liu and Pierre Baldi and D. Whiteson},
  title = {{Fast multi-geometry calorimeter simulation with conditional self-attention variational autoencoders}},
  year = {2024},
  eprint = {2411.05996},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.05996},
}
Figure