Introduction
In high-energy physics (HEP), identifying specific neutrino interactions is a fundamental challenge. Experiments like the Deep Underground Neutrino Experiment (DUNE) utilize Liquid Argon Time Projection Chambers (LArTPCs) to produce high-resolution, sparse 2D pixelated "photographs" of subatomic collisions. Distinguishing between electron neutrino charged current ($\nu_e$ CC), muon neutrino charged current ($\nu_mu$ CC), and neutral current (NC) interactions is critical for measuring neutrino oscillation parameters.
Traditionally, physicists have relied on hand-engineered features or Convolutional Neural Networks (CNNs). While CNNs are effective, they often function as "black boxes," providing classifications without physical justification. This research explores the application of Vision-Language Models (VLMs) to bridge this gap, combining visual pattern recognition with the reasoning capabilities of Large Language Models (LLMs) to provide "explainable physics."
Methods and Architecture
The researchers utilized LLaMA 3.2 Vision 11B, a multimodal model capable of simultaneous image and text processing. To adapt this general-purpose model to specialized physics data efficiently, the team employed QLoRA (Quantized Low-Rank Adaptation). This technique allows for fine-tuning by freezing the base model and training a small set of 4-bit quantized adapter weights, making the process feasible on modest hardware.
As shown in Figure 1, the model is provided with a system prompt describing the physical characteristics of different interactions (e.g., "fuzzy electron showers" for $\nu_e$ CC or "long tracks" for $\nu_mu$ CC). The input consists of dual-view pixel maps (XZ and YZ planes). To ensure machine-readable outputs, a phrasal constraint was implemented, forcing the model to begin its response with a specific prefix followed by the classification label.
Dataset and Simulation
The study utilized a simulated dataset of 190,000 labeled events generated via GENIE and GEANT4 to mimic LArTPC detector readouts. Each event is represented by two $512 \times 512$ grayscale images. The dataset includes three primary interaction classes: $\nu_e$ CC, $\nu_mu$ CC, and NC. The VLM was compared against a Siamese CNN baseline (3.4M parameters), a standard architecture for processing multi-view detector data.
Results and Performance
The fine-tuned VLM significantly outperformed the CNN baseline across all evaluated metrics. The VLM achieved an overall accuracy of 0.87, compared to 0.68 for the CNN. Notably, the VLM showed superior performance in discriminating Neutral Current (NC) events, which are traditionally difficult to distinguish from signal events.
As illustrated in Figure 4, the VLM achieved an Area Under the Curve (AUC) of 0.97 for both $\nu_mu$ CC and $\nu_e$ CC, and 0.95 for NC events, consistently exceeding the CNN's performance (AUC 0.90–0.94). This suggests that the VLM's ability to integrate semantic physical descriptions from the system prompt enhances its classification accuracy on sparse data.
| Metric | LLaMA 3.2 Vision | CNN Baseline |
|---|---|---|
| Accuracy | 0.87 | 0.68 |
| Precision | 0.87 | 0.78 |
| Recall | 0.87 | 0.78 |
| AUC-ROC | 0.96 | 0.93 |
Discussion and Limitations
While the VLM offers superior accuracy and the potential for natural-language rationales (e.g., describing a "fuzzy shower"), it introduces significant computational overhead. The VLM requires approximately 25.4 GB of VRAM and 3.3 seconds per sample for inference. In contrast, the CNN baseline requires only 1.0 GB of VRAM and processes samples in 20 milliseconds. This latency makes the current VLM approach unsuitable for real-time "online" triggers but highly valuable for "offline" analysis where interpretability and precision are paramount.
Conclusion
This work demonstrates that general-purpose Vision-Language Models can be successfully adapted to highly specialized scientific domains. By outperforming domain-specific CNNs, VLMs prove to be powerful backbones for physics analysis. Future research will focus on model distillation and the development of domain-specific foundation models to reduce computational costs while maintaining the interpretability benefits of the VLM approach.
Summary updated