Do Noisy Lensing Maps Still Hide Void Information? A Region-Decoupled Test

Cosmologists compress maps of the matter distribution into summary numbers and then infer parameters such as the matter density $\Omega_m$ and the fluctuation amplitude $\sigma_8$ from them. The standard summary is the power spectrum, which counts pairs of points at each separation. It is complete only for Gaussian fields, so it misses the clumps, filaments and empty voids that gravity builds. Neural networks trained directly on weak-lensing maps recover some of this missing information. Weak-lensing maps are reconstructions of projected mass from the slight distortion of background galaxy shapes. One saliency study found something odd about how these networks use the map. On clean simulated maps the network draws most of its evidence from underdense voids. Once realistic galaxy-shape noise is added, its attention moves almost entirely to the brightest mass peaks Interpreting deep learning models for weak lensing. That result is usually read as "noise destroys void information." It has a second reading that nobody has tested. Peaks are rare, high-signal regions, and they may dominate the training gradient, so the network never learns the weaker but still recoverable void signal.

This note proposes a direct test of that second reading, together with a method that would exploit it if it is true. The method splits each noisy map into regions defined from the observed data alone: peaks, voids and intermediate terrain. It trains a separate compressor for each region, freezes them all, and only then learns how to combine their outputs into a posterior over cosmological parameters. If this recovers information that a jointly trained network leaves unused, the "attribution inversion" is an artefact of how the network was optimized. That would change how full-field lensing analyses should be trained. If it recovers nothing, we will have shown cleanly that noise really erases void information, which is also worth knowing for survey design.

Core Claim

The claim is that, at the simulation budgets surveys can afford, joint end-to-end training of a field-level compressor under-uses low-amplitude regions of noisy lensing maps. The prediction is that compressors trained on one region at a time and combined only after freezing will give tighter, still calibrated posteriors than a single network trained on the whole map with the same budget. The gain should be concentrated in the void-derived summary, and it should shrink as the budget grows. The proposal stands or falls on one diagnostic. We freeze the summary produced by the best joint network and ask whether appending a separately trained void summary tightens the posterior. Define $t_J$ as the joint network's summary vector, $t_V$ as the void-region summary, and $\theta$ as the cosmological parameters. The quantity of interest is

$$ \Delta I = I(\theta; t_J, t_V) - I(\theta; t_J), $$

where $I$ is mutual information. We estimate it with the variational lower bound that is already used to train lensing compressors. A $\Delta I$ that is significantly positive means the joint network left void information on the table.

How this relates to existing work

Three lines of work motivate the question.

The saliency study above supplies the empirical puzzle. Its 18-layer network beats a combination of power spectrum, peak counts and Minkowski functionals by 20% in contour area even under shape noise. Its attribution moves from voids (69–89% of attribution when clean) to peaks above $3\sigma_\kappa$ (36–68% when noisy). It never asks whether the void information is gone or merely unused.

Cosmic-web environment studies split three-dimensional simulations into voids, walls, filaments and nodes using fixed thresholds on the tidal tensor, the second-derivative matrix of the gravitational potential. They then compute one power spectrum per environment, which tightens neutrino-mass constraints about 15× and $\Omega_m$ constraints about 8× in real space Cosmology with cosmic web environments I. Real-space power spectra, and up to 5.5× in redshift space Cosmology with cosmic web environments. These studies show that splitting by region exposes information. They use only two-point statistics within each region, work with noiseless dark-matter fields, and rely on segmentations that cannot be computed from an observed noisy map. We keep the region split but learn non-Gaussian summaries inside each region from noisy, observable maps.

Neural marked statistics Interpretable Equivariant Marks for Contrastive Cosmological Inference learn a reweighting of the whole density field. They are trained against parameters while explicitly subtracting what the power spectrum already explains. That residualization idea is what we borrow for the diagnostic above. The marks are applied globally rather than per region, and their advantage fades beyond $k \approx 0.3\ h\,\text{Mpc}^{-1}$.

The method comes from two places. From the simulation-based-inference benchmark Simulation-Based Inference Benchmark for Weak Lensing Cosmology we take its two-stage pipeline. A ResNet-18 is trained by variational mutual-information maximization to compress tomographic maps into six numbers. It is then frozen, and a normalizing flow learns the likelihood of those numbers. This is called neural likelihood estimation: a learned density over summaries replaces an analytic likelihood. Our change is to apply this per region and add a combination stage. From medical imaging we take two things. One is mask-weighted pooling: the segmentation mask weights the pooling of intermediate feature maps instead of zeroing input pixels. It is used in Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling and Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability. Because each encoder still sees surrounding pixels through its receptive field, it avoids the lost-context problem of hard cropping. The other is the logic of Havaei et al.: learn features where the rare signal is not outvoted, then calibrate on the natural distribution. In our setting the "majority class" is the high-signal peak pixels.

One paper partly cuts against the proposal. The same simulation-based-inference benchmark reports that its jointly trained compressor, with 101,000 simulations, already matches the posterior of explicit full-field Hamiltonian Monte Carlo sampling on log-normal maps. On that simulator, then, there may be no room left at large budgets. We take this seriously and design around it, as described below.

Research Program

The pipeline has three pieces.

The first is observable region definition. Each noisy convergence map is smoothed at a few scales. Pixels are assigned to peak, void or intermediate regions by thresholds on the smoothed signal-to-noise map. The thresholds are fixed in units of the noise level, so the masks are computable on real data. A soft, differentiable version uses sigmoid thresholds with learned centers. It is an ablation, not the default, because learned masks reintroduce coupling between regions.

The second is Stage 1. One ResNet-18-sized encoder per region sees the full map. Its final feature maps are pooled with that region's mask, and it is trained alone with the mutual-information objective to output a short summary vector. A fourth encoder trained on the unmasked map supplies $t_J$.

The third is Stage 2. All encoders are frozen. A single normalizing flow learns the likelihood of the concatenated summaries, and posteriors come from standard sampling.

The analysis has three parts. The first is the $\Delta I$ diagnostic. The second is a sweep over simulation budget to see where any gain lives. The third is a re-run of the attribution analysis on the combined model, to check whether void attribution returns under noise.

Baselines

The naive reference is the classical combination of power spectrum, peak counts and Minkowski functionals, fed to the same normalizing-flow likelihood estimator. It anchors how much any neural method adds. On the ray-traced maps the cited result is that a deep network shrinks the $(\Omega_m, \sigma_8)$ contour by 20% relative to this combination under realistic shape noise Interpreting deep learning models for weak lensing.

The strongest existing method is the jointly trained ResNet-18 compressor with the mutual-information objective followed by neural likelihood estimation Simulation-Based Inference Benchmark for Weak Lensing Cosmology. It is strongest because it is the same two-stage machinery we use, minus the region split. On log-normal LSST Year 10 maps it reaches the explicit-sampling posterior quality, at a classifier two-sample-test threshold of 0.6, with 101,000 simulations versus 630,000 for explicit Hamiltonian sampling. No reported contour-area or figure-of-merit number for this compressor was found on Lacuna, so we will measure it ourselves. For fairness it receives the same total parameter count as our ensemble of region encoders: we widen it to match. It also gets the same simulations, the same number of optimizer steps summed across encoders, and the same hyperparameter search budget per configuration. Beating it means a tighter $(\Omega_m, \sigma_8)$ contour at equal budget, with calibrated coverage on held-out simulations and no worse two-sample-test score. On the ray-traced maps the target is an improvement beyond the 20% that end-to-end networks already achieve over classical statistics.

Two ablations isolate the key ingredients. The decoupling ablation uses the identical set of region encoders with mask pooling but trains them together under one shared loss. Any gain we see must survive this, or it comes from architecture, not from separate optimization. The region ablation replaces physical masks with random masks of matched area fraction and smoothness, which tests whether the region semantics matter.

Why not just…?

Why not just train the end-to-end network longer, wider, or on more simulations? This is the most serious threat. On log-normal maps the cited benchmark already finds that a joint compressor saturates the explicit full-field posterior at 100,000 simulations, so whatever is "lost" may only be lost at small budgets. We therefore do not claim an asymptotic information gain on log-normal maps. The budget sweep measures the gap to the explicit-sampling ceiling at 1,000, 5,000, 20,000 and 100,000 simulations for both methods. If the region-decoupled model closes the gap faster, the contribution is sample efficiency. If not, we report that honestly. The claim of real information gain is reserved for ray-traced maps, where no explicit-likelihood ceiling exists and where the attribution inversion was observed.

Why not just give one network the region masks as extra input channels? That gives the network the region information while keeping one coupled optimization. If gradient domination by peaks is the cause, this baseline should fail in the same way as the plain network. If it matches our method, the useful ingredient is the region information, not the decoupling, and the joint-training ablation above would show the same. We include it as a further baseline, budget-matched.

Why not just add handcrafted void statistics such as void counts or Minkowski functionals of the negative-convergence region? These are cheap and interpretable. If they reproduce $\Delta I$, a neural void encoder is unnecessary. The evidence that handcrafted summaries leave information behind is the 20% gap above. That gap was measured globally, though, not for void-specific statistics, so we test it directly. We substitute a handcrafted void vector for $t_V$ in the $\Delta I$ diagnostic and compare.

Why not just use a global learnable scattering transform? Learnable wavelets are parameter-efficient and strong at small budgets. On CAMELS maps with 1,000 training samples, they reach 6.32% error on $\Omega_m$ versus 10.07% for a CNN Learnable wavelet neural networks for cosmological inference. They therefore compete directly in the low-budget regime where we expect our gain. We run it as an additional budget-matched baseline. We also test a region-pooled variant, which pools wavelet coefficients under each mask, to separate "better filters" from "region decoupling."

Proposed Experiments

The primary testbed is the simulated LSST Year 10 tomographic convergence maps from the simulation-based-inference benchmark. These are $256\times256$ pixel maps over 10×10 square degrees in five redshift bins, made by a differentiable log-normal model. They allow cheap budget sweeps and provide an explicit-sampling ceiling. The realism test uses ray-traced convergence maps across 101 $(\Omega_m, \sigma_8)$ cosmologies with and without shape noise, as in the saliency study. A secondary test applies the same recipe to three-dimensional Quijote fields, using tidal-tensor environments as the regions. There we ask whether learned per-environment summaries beat per-environment power spectra at the same simulation count.

The metrics are $(\Omega_m, \sigma_8)$ contour area, coverage calibration from rank statistics on held-out simulations, parameter mean-squared error, and, on log-normal maps, the classifier two-sample-test score against explicit sampling. All are reported with seed-to-seed spread over at least five seeds.

The decisive checks are three. First, $\Delta I$ must be positive and exceed its seed-to-seed spread on noisy ray-traced maps while staying near zero on noiseless ones; that pattern is what "unused, not destroyed" predicts. Second, the gain must survive the joint-training ablation and vanish under random masks. Third, attribution recomputed on the combined model under noise must put substantially more weight on void pixels than the 36–68% peak-dominated pattern reported for end-to-end networks, while coverage remains calibrated. If $\Delta I$ is indistinguishable from zero under noise, we conclude that shape noise genuinely erases void information at these map resolutions and redirect effort toward the peak regions.