Keeping Fine-Scale Latents Alive to Quarantine Baryonic Physics in CAMELS

Cosmologists want to read two numbers off maps of the simulated universe: $\Omega_m$, how much matter there is, and $\sigma_8$, how clumpy it is. The obstacle is that small scales are shaped by poorly understood "subgrid" recipes for supernova and black-hole feedback, and different simulation codes implement these recipes differently. A model trained on one code gives biased answers on another. The standard remedy is to throw small scales away. We propose a different route: a hierarchical variational autoencoder, meaning a generative model that compresses a map into a stack of random latent codes at several spatial resolutions and learns to reconstruct the map from them. We then train it so that the fine-resolution codes are forced to carry the small-scale, feedback-driven structure. If the fine codes do this job, the coarse codes no longer need to encode feedback, and inference from the coarse codes alone should transfer across simulation codes without a hard scale cut.

The obstacle is a known training failure. Posterior collapse is the case where the model learns to ignore some latent codes and the decoder reproduces the data without them. In a multi-scale model, the fine codes are the likely casualties, because large high-contrast density peaks dominate the reconstruction loss. Our central claim is that fine-scale collapse is not just an efficiency problem. It causes feedback information to leak into the coarse codes, and that leakage is what makes latent-based cosmological inference fragile across codes. We test this claim with a scale-resolved KL controller. The KL term is the penalty that pulls each latent code toward an uninformative prior; the controller adjusts that penalty separately for each resolution. Its targets are stated in physical units, as a tolerated fraction of residual power in each wavenumber band, rather than as hand-picked information budgets.

Core Claim and What Is New

The work at this intersection so far leaves out the piece we need. Diffusion models on CAMELS reconstruct dark matter fields in full pixel space. Debiasing with Diffusion conditions a variational diffusion model on stellar maps, and 3D Reconstruction of Dark Matter Fields with Diffusion Models does 3D conditional reconstruction and reports lost power above $k \approx 0.3\,h/\text{Mpc}$. Neither has a compact latent space partitioned by scale. Detecting Model Misspecification with Scale-Dependent Normalizing Flows motivates us most directly. It compresses smoothed CAMELS maps with a deterministic ResNet and shows that the codes agree on large scales but diverge on small ones. It diagnoses the problem through smoothing sweeps but builds no generative model that separates the two regimes.

On the machine learning side, NVAE supplies the architecture: a deep hierarchical autoencoder stable across 36 latent groups. It was tested only on natural images, and the research file characterizes it as using the same KL schedule at every depth. ControlVAE supplies the control idea: a proportional-integral (PI) controller that adjusts one global KL weight toward a KL target. Its stated weakness is that the target must be hand-tuned, measured in nats. Posterior Collapse in Linear Conditional and Hierarchical VAEs constrains us by establishing that hierarchical models are prone to partial, layer-specific collapse.

What is new here has three parts. First, we treat per-scale collapse as a cause of cross-code bias and measure it. Second, we use one controller per resolution tier. Third, each controller's set point is expressed in the physics of the problem: band-limited reconstruction error relative to the true power spectrum.

Research Program

We train NVAE-style hierarchical VAEs on 2D total-matter and gas-density maps from the CAMELS Multifield Dataset. 2D maps keep memory manageable; the file notes that dense multi-scale 3D architectures scale poorly. The latent groups are organized into $S$ resolution tiers. Each tier $s$ is assigned a wavenumber band $B_s$ bounded by that tier's grid resolution.

For each tier we measure a band residual ratio,

$$R_s = \frac{\sum_{k \in B_s} P_{x-\hat{x}}(k)}{\sum_{k \in B_s} P_x(k)},$$

where $P_x$ is the power spectrum of the true map and $P_{x-\hat{x}}$ is the power spectrum of the reconstruction residual.

Each tier has its own KL weight $\beta_s$. We update it with a PI rule on the error $e_s = R_s - R^\*_s$:

$$\beta_s(t+1) = \beta_s(t) - K_p\, e_s(t) - K_i \sum_{\tau \le t} e_s(\tau),$$

clipped to $[\beta_{\min}, 1]$. Here $K_p$ and $K_i$ are gains shared across tiers, and $R^\*_s$ is the tolerated residual fraction. When a band is under-reconstructed, its KL penalty is relaxed, so the corresponding latents are allowed to carry information. We set $R^\*_s$ from a quantity cosmologists already understand: the scatter of $P(k)$ across CAMELS realizations that share the same parameters. That removes ControlVAE's free set point.

Latent tiers do not map cleanly onto frequency bands, so we also measure a band-attribution matrix. For each tier, we resample it from the prior while holding the other tiers fixed, then record how much power changes in each band. Two diagnostics then carry the scientific claim. The first is leakage: how well a small held-out regressor predicts the feedback parameters ($A_{\text{SN1}}, A_{\text{SN2}}, A_{\text{AGN1}}, A_{\text{AGN2}}$) from the coarse latents alone. The second is cross-code bias: train cosmology inference on the coarse latents of IllustrisTNG maps, then evaluate on SIMBA and Astrid maps.

Baselines

The naive reference point is a CNN that regresses $\Omega_m$ and $\sigma_8$ directly from full-resolution maps, as in the CAMELS data release and the Multifield Dataset paper. The file reports these networks as achieving "percent-level accuracy" within a single suite and says they degrade across codes. The file gives no specific cross-code error figure for this CNN on Lacuna. We retrain the CNN ourselves on the same IllustrisTNG maps with the same hyperparameter search budget. Beating it means lower bias on SIMBA and Astrid at comparable in-suite accuracy.

The strongest existing approach to misspecification is scale cutting. On the inference side, that means CNN compression of Gaussian-smoothed maps, in the style of the scale-dependent flows paper. On the detection side, it means that paper's own evidence test. It reports that SIMBA and IllustrisTNG gas maps fall below a 20% percentile rank at small scales when the model is trained on Astrid. We reproduce its smoothing sweep and set our method against the best smoothing scale chosen on validation data. Two results count as beating this baseline. One is matching its cross-code bias at a smaller posterior width, which shows we kept information the cut threw away. The other is detecting misspecification at least as sharply using the evidence of the fine latents alone.

Within the VAE family, we compare three alternatives against the proposed per-band controller:

  • the same architecture with a single global warm-up, the NVAE-style default;
  • global cyclical annealing;
  • global ControlVAE with a tuned KL target.

The file reports ControlVAE beating cyclical annealing on Switchboard, with perplexity 8.81 versus 17.81. That result is from text, so it only motivates the comparison; it proves nothing about maps.

Two ablations isolate the key ingredients:

  • Per-tier free bits. A fixed minimum KL for each tier, with no feedback loop. This tests whether control matters at all.
  • Per-tier controller with nat-valued targets. This tests whether physics-derived set points matter.

All variants share the same architecture, data, number of optimizer steps, and search budget over gains or floors.

Why not just…?

Why not just use a diffusion model, which has no bottleneck to collapse? Diffusion models reconstruct fields well; Debiasing with Diffusion reports a cross-correlation above 0.9 across most scales. But they give no compact, scale-partitioned summary to run inference on, and the 3D variant lost small-scale power at $256^3$ resolution. Our question concerns where feedback information sits in a latent space, and a model without one cannot answer it. We therefore compare power spectra against a diffusion model only as a check on reconstruction quality, not as an inference competitor.

Why not just cut small scales, as SimBIG with the wavelet scattering transform does? This is the practical default, and it works. In that paper the cut at $j_{\min}=2$ still tightened $\sigma_8$ constraints by 1.8x relative to perturbation theory. The cost is that the cut discards non-linear information. The smoothing baseline above settles the question: if our method shows no smaller posterior width at equal bias, the cut wins and we will report that.

Why not just use a global cyclical or ControlVAE schedule? A single KL weight cannot respond differently to coarse structures with large variance and diffuse fine structures. The prediction is specific: under a global schedule the fine tiers collapse (near-zero KL, flat rows in the attribution matrix) and leakage rises. If global schedules keep the fine tiers active and leakage stays low, the per-tier machinery is unnecessary and the leakage finding stands by itself.

Why not just set per-tier free bits? This is the real threat. A fixed KL floor per tier might keep the fine latents alive with no controller. Free bits cannot guarantee that the preserved information is the band-appropriate information, and the floor still has to be chosen in nats. If the free-bits ablation matches our method on both leakage and cross-code bias, we will conclude that the contribution is the scale-partitioned latent design and its diagnostics, not the controller.

Why not just reweight the reconstruction loss by band? This is the analogue of lesion size reweighting. That paper found that strict inverse weighting destabilized training and that only a bounded scheme worked. We include bounded band reweighting as a further ablation. It raises fine-scale fidelity but does not stop the model from routing feedback information through the coarse latents, so leakage is the metric that separates it from our method.

Decisive Checks

The study rests on three measurements, all trained on IllustrisTNG maps and evaluated on held-out IllustrisTNG, SIMBA and Astrid maps:

  1. Collapse causes leakage. Across all VAE variants, fine-tier KL should be anti-correlated with how well coarse latents predict AGN and supernova parameters.
  2. Leakage causes bias. Across the same variants, leakage should predict cross-code bias in $\Omega_m$ and $\sigma_8$ inferred from coarse latents.
  3. The best variant beats scale cutting. It should achieve bias no worse than the best smoothing cut, with narrower posteriors, and its fine-latent evidence should flag SIMBA as out-of-distribution at least as reliably as the scale-dependent flow does.

If the first relationship does not hold, meaning fine-scale collapse does not predict leakage, the central hypothesis is false regardless of how well any controller performs.