Contrast-Weighted Distillation Search for Leptonic–Hadronic Emulators

Gamma rays above 100 TeV can come from two kinds of particle accelerators. In the leptonic case, fast electrons scatter low-energy photons up to high energy, a process called inverse Compton scattering. In the hadronic case, fast protons hit interstellar gas and make neutral pions, which decay into gamma-ray pairs. These two cases often produce nearly identical spectra. The best way to tell them apart is to model a source's spectrum together with how its apparent size changes with energy, since cooling electrons make a source look smaller at higher energy. That joint model requires running a physics simulator for particle transport, radiation, and the instrument response inside a likelihood fit, and the simulator is too slow for routine use. The usual fix is a neural emulator, a small network trained to imitate the simulator. But an emulator that is accurate on average can still be wrong in exactly the few energy bins and pixels where the two hypotheses differ. When the hypotheses barely differ, that kind of error alone can decide which mechanism the analysis prefers.

We propose to design such emulators with distillation-guided architecture search. This family of methods trains each block of a candidate network to copy the matching intermediate output of a "teacher" and ranks architectures by those block-level errors. Here the teacher is the physics simulator itself, and its natural stages serve as the blocks: the particle distribution after cooling and diffusion, the emissivity from each radiative process, and the projected counts maps after the instrument response. The new ingredient is the ranking score. Each block's error is pushed through the rest of the simulator to the observed data, and it is weighted by how strongly the leptonic and hadronic predictions differ in each bin relative to the instrument noise. The search therefore selects emulators whose mistakes do not move the leptonic-versus-hadronic Bayes factor, the ratio of evidence for the two models, rather than emulators with the lowest average error. With these emulators we aim to run joint spatial-spectral model comparison across LHAASO sources and report, for each source, whether the degeneracy is real or only an artifact of separate fitting.

Core Claim

The claim has two parts. The first is about method. The simulator's stages give a block decomposition in which the main weakness of block-wise ranking can be fixed in a principled way. That weakness is that block errors are summed as if each block's mistakes were independent of the others. The research file identifies a real tension here. DONNAv2 ranks networks by a plain sum of block mean-squared errors against a teacher. Broadband emission, however, is not separable this way: the X-ray synchrotron emission and the TeV inverse Compton emission are tied together through one shared electron distribution and one magnetic field (1ES 1959+650, joint spectral fitting; Discovery of VHEγ-rays from the blazar 1ES 1215+303 with the MAGIC telescopes and simultaneous multi-wavelength observations).

When the blocks are the simulator's own stages, that coupling sits entirely in the intermediate outputs those stages pass to each other. An error in the electron-distribution block reaches the observables only through the downstream radiative and instrument stages. Its effect on the data is therefore given by those stages' Jacobian, the matrix of how each output responds to a small change in each input, and this is computable. We define the ranking proxy for architecture $a$ as

$$\hat{E}(a) = \sum_{k} \mathbb{E}_{\theta}\Big[\big\| W_{\theta}\, J_{k,\theta}\, (s^{a}_{k} - t_{k,\theta}) \big\|^{2}\Big].$$

In this expression, $\theta$ is a physical parameter set drawn from the prior. $t_{k,\theta}$ is the simulator's output at stage $k$, and $s^{a}_{k}$ is the student block's output for the same stage. $J_{k,\theta}$ is the Jacobian of the remaining simulator stages, from stage $k$ to the binned counts. $W_{\theta}$ is a diagonal weight that, in each energy bin and pixel, scales up error in proportion to $|\mu_{\text{lep}} - \mu_{\text{had}}|/\sigma$. Here $\mu_{\text{lep}}$ and $\mu_{\text{had}}$ are the best-matching leptonic and hadronic predictions for $\theta$, and $\sigma$ is the expected Poisson-plus-background noise in that bin. The weight follows from the log-likelihood ratio: to first order, an emulator error changes the log Bayes factor by its projection onto this whitened contrast direction.

The second part of the claim is scientific. Emulators chosen this way make joint 3D (two sky coordinates plus energy) model comparison cheap enough to apply across a catalog. They also keep the Bayes-factor bias from emulation small compared with the statistical width of the Bayes factor itself.

Relation to Prior Work

Three distillation-guided search methods supply the machinery. DNA Family shows that supervising each block against a teacher shrinks a search space of more than $10^{17}$ candidates into tractable sub-problems. It raises the rank correlation between the search score and stand-alone accuracy to Kendall's τ = 0.64 on ImageNet. DONNAv2 trains candidate blocks for a single epoch against a reference network and uses the summed block errors in place of an accuracy predictor. DisWOT goes further and scores untrained students by how well their attention maps and relations between samples match the teacher's.

We reuse the block-wise training of DNA and DONNAv2 and the multi-objective evolutionary search of DONNAv2. We depart from all three in two ways. Our teacher is a physics simulator whose intermediate quantities have physical meaning, not a larger network. Our ranking target is preservation of a decision between two hypotheses, not classification accuracy. The DNA and DONNAv2 results motivate the approach, because they show block supervision can rank architectures cheaply. The additive assumption in DONNAv2 constrains the design, because summing raw block errors ignores how errors grow as they pass through later stages, and that is exactly what our Jacobian weighting addresses. DisWOT's training-free proxies rely on Grad-CAM attention maps over images. They have no obvious counterpart for spectra that span many decades in energy, so we treat them only as a cheap filter to test, not as the main method.

The astrophysics papers supply both the motivation and the test cases. Energy-dependent shrinkage is the clearest discriminator available. In 1LHAASO J0056+6346u the source extent falls from about 0.34° (1–25 TeV) to 0.21° (>100 TeV), yet nearby molecular gas still leaves a hadronic origin open. In CTA1 the extent falls from about 0.23° to about 0.17° and the data are modeled leptonically with B ≈ 4.5 μG. In both papers, morphology and spectrum are fitted in separate steps.

LHAASO J2108+5157 fits one-zone models with naima/jetset. A leptonic model with B < 1.2–1.9 μG and a hadronic model with two molecular clouds both survive. IC 443 contains a halo whose roughly 20 TeV cutoff is compatible with either mechanism. Our proposal does not claim that machine learning can resolve a degeneracy that is truly present in the data. It claims that a joint likelihood which has never been evaluated, because doing so was too costly, can now be evaluated without emulator error biasing the result.

Research Program

The teacher is a forward model written for this project, with two branches and three stages. Stage one takes injection parameters and returns a particle distribution $N(E, r)$ that has undergone diffusion and cooling (synchrotron and Klein–Nishina inverse Compton losses for electrons, a power law with cutoff for protons). Stage two returns emissivity per photon energy at each position. On the leptonic side this is inverse Compton scattering off the cosmic microwave background and infrared fields, plus synchrotron emission, checked against naima in the one-zone limit. On the hadronic side it is pion decay weighted by gas density taken from MWISP CO and CGPS HI maps. Stage three projects the emission along the line of sight and applies WCDA and KM2A point-spread functions and energy resolution, giving counts maps in the energy slices those papers use.

The student has one searchable block per stage. The stage-one block chooses among multilayer networks of different width and depth, with and without Fourier features of log-energy. The stage-two block uses conditioned kernel networks. The stage-three block chooses among convolutional and spectral-operator designs. Following DNA, each block's search space is a few thousand candidates. Blocks are trained against the matching teacher stage, and assembled networks are ranked by $\hat{E}$. Following DONNAv2, the search is run as a two-objective problem trading $\hat{E}$ against inference cost.

Once emulators are selected, we wrap them in nested sampling for each source and hypothesis. The pipeline reports three things: the joint Bayes factor, the emulation bias (estimated on a subset of points re-run through the true simulator), and the counterfactual measurement that would most widen the gap between the hypotheses. For example, that might be a deeper X-ray upper limit or more exposure above 100 TeV.

Baselines

The do-nothing reference is the current practice, reproduced on the same sources. This means a one-zone naima spectral fit together with a separate energy-sliced morphology fit, as in 1LHAASO J0056+6346u and CTA1. It shows what joint modeling adds. Beating it means that on injected sources, where the true mechanism is known, joint evidence identifies the mechanism correctly more often than the decoupled fits do at equal significance. It also means that on real sources our pipeline recovers the published leptonic parameters within their quoted errors: B ≈ 4.5 μG and an electron cutoff near 373 TeV for CTA1, and B < 1.9 μG for J2108+5157.

The strongest competitor is a fixed-architecture emulator. This is a well-tuned deep multilayer network for the spectral stages and a U-shaped convolutional network for maps, trained end to end on the same simulator outputs with the same number of simulator calls and the same GPU-hour budget as our full search plus training. It also receives an equal hyperparameter-tuning budget. It is the real threat because emulators of smooth physical functions are often easy to build. We found no reported result on Lacuna for neural emulators of naima or jetset, or of joint spatial-spectral gamma-ray models, so its performance must be measured here. Beating it means a lower Bayes-factor bias at equal inference cost, or equal bias at clearly lower cost.

The strongest existing search method is DONNAv2-style ranking by plain summed block mean-squared error over the same blocks and the same search space. Its reported strength, on vision rather than physics tasks, is an equivalent of DNA's Kendall's τ = 0.64 on ImageNet (DNA Family). We measure τ between each proxy and stand-alone Bayes-factor bias on 200 fully trained architectures. Beating it means a higher τ and a better final emulator.

Three ablations isolate the key ingredients. The first keeps Jacobian propagation but drops the contrast weight $W$. The second keeps $W$ but drops the Jacobian. The third places block boundaries at arbitrary network depths rather than at simulator stages. Together they test whether the gain comes from the contrast weighting, from error propagation, or from the physical choice of blocks.

Why not just…?

Why not just run MCMC with the real simulator? For one-zone spectral fits this is affordable, and there we expect little or no gain. It is a real threat, so we will measure wall-clock time per evidence estimate for the joint 3D model with and without emulation. If the direct simulator turns out cheap enough for catalog-scale nested sampling, the method's contribution shrinks to amortization, meaning the training cost is paid once and reused across sources. We will report that honestly. The case for emulation rests on the time-dependent diffusion-cooling and projection stages, not on spectra alone.

Why not just train one large fixed emulator? That might be enough. The fixed-emulator baseline above settles the question directly. If it matches our searched emulators on Bayes-factor bias at equal cost, the conclusion is that search is unnecessary in this domain. The surviving contribution would then be the contrast-weighted error measure, used as a training loss and as an acceptance test for any emulator.

Why not just put the contrast weight into the training loss and skip the search? This isolates whether weighting matters for training or for ranking. We will run a fixed architecture trained with the $\hat{E}$ loss. If it closes the gap, we will say so. We expect it to help but not fully close the gap, because the weight changes from source to source, and an architecture that lacks capacity where the contrast is concentrated, such as around sharp cutoffs or compact cores, cannot be rescued by reweighting the loss.

Why not just wait for neutrinos? As the research file notes, most Galactic PeVatron candidates have no significant neutrino counterpart. Predicted fluxes for even very bright flares fall below current sensitivity (1ES 1959+650). Waiting would leave most of the 90 sources in the First LHAASO Catalog uncharacterized.

Proposed Experiments

The first experiment is injection and recovery on synthetic LHAASO-like data. We draw mechanism and parameters from priors centered on the CTA1, J0056+6346u, and IC 443 halo fits, and we embed the simulated sources in the observed Galactic diffuse emission background, with spectral index about −2.67 in the inner plane. Metrics are Kendall's τ of each ranking proxy against true emulator-induced Bayes-factor bias, the bias in log Bayes factor measured against true-simulator evidence on a held-out subset, how well calibrated mechanism recovery is, and inference cost.

The decisive check is a scatter plot of per-source emulation bias against the statistical width of the log Bayes factor. The proposal succeeds if the searched emulators stay well below that width where the hypotheses are nearly degenerate, while the unweighted and fixed baselines do not.

The second experiment applies the validated pipeline to the real data in the cited papers: Crab, CTA1, J0056+6346u, IC 443, J2108+5157, and J0341+5258. For each source we report the joint Bayes factor, whether it differs from the decoupled-fit conclusion, and which counterfactual observation would most widen the leptonic–hadronic gap. A result in which the joint analysis keeps J2108+5157 degenerate but separates J0056+6346u would be informative. A result in which the joint analysis changes no conclusions would be equally publishable, as evidence that the degeneracy is physical rather than an artifact of separate fitting.