CaloDREAM – Detector response emulation via attentive flow matching

Luigi Favaro, Ayodele Ore, Sofia Palacios Schweitzer, Tilman Plehn

TL;DR

CaloDREAM is a new generative framework that uses Conditional Flow Matching and Transformers to simulate high-fidelity particle showers in calorimeters. By factorizing the problem into energy and shape components and employing "bespoke" ODE solvers, it achieves state-of-the-art precision and significant sampling speedups on datasets with up to 40,000 dimensions.

What Problem Does This Paper Address?

In high-energy physics, simulating how particles interact with detectors (like those at the Large Hadron Collider) is a massive computational bottleneck. The gold standard, GEANT4, is based on first-principles physics but is incredibly slow.

Machine learning researchers have turned to generative models to speed this up, but calorimeter data presents a "perfect storm" of difficulty:

  1. High Dimensionality: Modern detectors have tens of thousands of readout cells (voxels).
  2. Extreme Sparsity: Most cells contain zero energy, creating sharp discontinuities that traditional neural networks struggle to map.
  3. Precision Requirements: Simulations must be accurate at the percent level to be useful for discovering new physics.

While diffusion models have shown promise in handling these distributions, they are notoriously slow to sample from, often requiring hundreds of iterations to produce a single event.

Methods

The authors introduce CaloDREAM (Detector Response Emulation via Attentive flow Matching), which combines several cutting-edge techniques:

  • Conditional Flow Matching (CFM): Instead of standard diffusion, the authors use CFM. It learns a vector field that "pushes" a simple Gaussian distribution toward the complex data distribution along straight-line trajectories, making the learning process more stable and efficient.
  • Factorized Architecture: The model is split into two specialized networks:
  • Energy Network ("Transfusion"): An autoregressive transformer that predicts the total energy deposited in each detector layer.
  • Shape Network (ViT): A 3D Vision Transformer that takes the predicted layer energies and determines the spatial distribution (voxels) within those layers.
  • Latent Diffusion (laViT): For the most high-resolution datasets, the model doesn't work directly on voxels. Instead, it uses a Variational Autoencoder (VAE) to compress the 40,000-dimensional space into a smaller latent space, where the CFM model operates.
  • Bespoke Non-Stationary (BNS) Solvers: Rather than using a generic ODE solver (like Runge-Kutta), the authors train a small, specialized solver to "learn" the specific trajectories of their model. This allows them to generate high-quality samples in far fewer steps.

Datasets

The framework was benchmarked on the CaloChallenge 2022 datasets:

  • Dataset 2 (DS2): ~6,500 voxels representing a medium-granularity detector.
  • Dataset 3 (DS3): ~40,500 voxels representing a high-granularity detector. Both datasets consist of 200,000 electron showers simulated with GEANT4.

Key Insights and Ideas

  • Transformers are Sparsity-Friendly: By using Vision Transformers (ViTs) on "patches" of voxels, the model can focus on relevant energy clusters rather than getting bogged down by the vast empty spaces in the detector.
  • The Power of Factorization: Predicting layer energies first (the "causal" part of the shower) provides a strong conditioning signal for the shape network, significantly improving the physical consistency of the generated showers.
  • Optimizing the Solver, Not Just the Model: Most "fast" diffusion research focuses on distillation (training a new model to take fewer steps). This paper shows that keeping the model fixed and simply optimizing the solver (BNS) is a highly effective, lower-cost alternative.
  • Latent Space Trade-offs: While latent diffusion is necessary for high-dimensional data like DS3, it can struggle with very low-energy voxels. The authors found that a simple MeV-level energy threshold during post-processing fixes most of the resulting "sparsity" errors.

Key Results

  • High Fidelity: On DS2, the model achieved a classifier AUC score of ~0.53 (where 0.5 is a perfect "fooling" of the discriminator), indicating the generated showers are nearly indistinguishable from GEANT4 truth.
  • Scalability: The latent diffusion approach (laViT) successfully handled the 40k-dimensional DS3 dataset, maintaining high precision across complex physical observables like shower depth and width.
  • Massive Speedup: Using the Bespoke BNS solver, the model achieved high-quality results in just 4 to 8 function evaluations. Standard solvers typically require 50 to 100 steps to reach the same level of accuracy.
  • Hardware Performance: A single forward pass of the shape network takes roughly 17ms (DS2) to 84ms (DS3) on an NVIDIA H100 GPU.

Limitations and Future Work

  • Low-Energy Noise: The latent diffusion model tends to overestimate the energy in very low-energy voxels, which affects the "sparsity" distribution of the detector. While thresholding helps, a more principled fix within the VAE architecture would be ideal.
  • Bayesian Uncertainty: The authors noted that training Bayesian versions of these large transformer models (to estimate "how sure" the model is) is currently too computationally expensive for standard research budgets.
  • Generalization: The current study focuses on electron showers. Future work is needed to test the framework on more complex particles like hadrons, which produce much "messier" and more irregular showers.

Summary updated

Abstract

Detector simulations are an exciting application of modern generative networks. Their sparse high-dimensional data combined with the required precision poses a serious challenge. We show how combining Conditional Flow Matching with transformer elements allows us to simulate the detector phase space reliably. Namely, we use an autoregressive transformer to simulate the energy of each layer, and a vision transformer for the high-dimensional voxel distributions. We show how dimension reduction via latent diffusion allows us to train more efficiently and how diffusion networks can be evaluated faster with bespoke solvers. We showcase our framework, CaloDREAM, on datasets 2 and 3 of the CaloChallenge.
BibTeX
@inproceedings{favaro2025calodream,
  author = {Luigi Favaro and Ayodele Ore and Sofia Palacios Schweitzer and Tilman Plehn},
  title = {{CaloDREAM – Detector response emulation via attentive flow matching}},
  booktitle = {SciPost},
  year = {2025},
  doi = {10.21468/scipostphys.18.3.088},
}

Loading related content...

Figure