Trust Maps for Virtual Radar Over Radar-Blind Regions

Ground weather radar covers only about 250 km around each site, is blocked by mountains, and cannot be placed over open ocean. To fill these gaps, a growing line of work trains neural networks to turn geostationary satellite images into "virtual radar," meaning maps that look like the reflectivity a ground radar would have measured. These models are trained and scored where real radar exists, but they are needed most where it does not. In those places nobody can check the output, and the satellite inputs are known to behave differently: over the ocean, water vapor channels matter more and models trained on land degrade. This proposal asks for a per-pixel map that says how far to trust a virtual radar product in places with no ground truth.

The core idea comes from reconstruction-based anomaly detection. We train a generative model only on satellite imagery from radar-covered regions, then ask it to fill in masked pieces of new satellite scenes. Where it cannot fill them in accurately, the scene lies outside what the virtual radar model learned from, and the virtual radar there should be distrusted. The probe looks only at the satellite input. It needs no radar labels in the blind region and works with any virtual radar model. The claim to test is whether its inpainting error predicts where virtual radar is wrong under geographic shift, and whether it adds information beyond the synthesizer's own uncertainty.

Core Claim

Virtual radar models fail silently when moved to new geography. Their built-in uncertainty, such as disagreement among ensemble members or spread across diffusion samples, is tuned only on the regions they were trained on. We claim that a masked-inpainting probe trained on satellite data from radar-covered regions gives a confidence map that ranks synthesis error in held-out regions at least as well as synthesizer-side uncertainty, and that combining the two ranks error better than either alone. The second part is the decisive one. If the probe adds nothing once ensemble spread is known, the method is redundant, and we say so in advance.

Where the Directions Meet, and What Is Missing

Virtual radar synthesis has moved quickly. Early work treated it as plain image translation: U-Nets mapping multispectral imagery to precipitation for an operational product (Lebedev et al. (2019)), deep networks estimating composite reflectivity from Fengyun-4A (Deep Learning-Based Radar Composite Reflectivity Factor Estimations from Fengyun-4A Geostationary Satellite Observations), and the SEVIR benchmark, which pairs GOES-16 imagery and lightning data with NEXRAD vertically integrated liquid. Later work improved fidelity in three ways. MAFormer added attention modules for multi-scale structure. DiffSR refines a coarse transformer estimate with a conditional diffusion model, which is a generator that learns to turn noise into samples step by step; this recovers intense storm cores. Station2Radar anchors satellite estimates to rain gauges. All of these produce a field. None says where that field can be trusted, and Station2Radar's anchoring is unavailable over open water, where no gauges exist.

Two papers motivate this proposal directly by showing the shift is real. Radar Echo Reconstruction in Oceanic Area via Deep Learning of Satellite Data trained separate models on land, coast, offshore and sea data. It found that a mixed land, coast and offshore training set transfers to the ocean better than sea-only data, and that the inputs the model relies on shift toward water vapor offshore. Robust Synthesis Weather Radar from Satellite Imagery found that visible channels at dusk and night corrupt predictions badly enough that a dedicated day/night router is needed. That router is a hand-designed detector for one known shift. We want a learned detector for shifts nobody has named in advance.

The method comes from anomaly detection. Lift, Map, Detect (LMD) masks an image with a checkerboard, inpaints it with a diffusion model trained only on in-distribution data, and scores out-of-distribution status by reconstruction distance. It reports a mean ROC-AUC (area under the receiver operating curve) of 0.907 across 12 dataset pairs. It also explains why we do not use likelihood: on CIFAR-100 versus SVHN, likelihood-based detectors score an ROC-AUC of 0.193, while inpainting distance scores 0.985. Two other papers constrain the design. DRÆM trains a second network to turn reconstruction residuals into a localization map in one forward pass, which suggests a cheaper variant. DeScarGAN removed the top skip connection of its U-Net so the generator cannot copy anomalies straight through. That warning applies to any reconstruction probe we build.

We differ from LMD in three ways. The data are physical brightness temperatures rather than photographs. The target is not "is this image foreign" but "will a downstream model err here." The masks run over spectral channels as well as space. We differ from DiffSR and its relatives because they make the field better, while we estimate where any such field is unreliable.

Research Program

The probe is a conditional inpainting model over satellite inputs alone: SEVIR's visible, water vapor and longwave infrared channels plus lightning, or the corresponding Himawari or GOES ABI bands. It is trained only on scenes inside radar coverage. We use two kinds of mask. Spatial masks follow LMD's alternating checkerboard. Channel-group masks hide, for example, the water vapor channels given the infrared and lightning channels. The channel masks matter because the documented ocean shift is a change in how channels relate to each other, not only a change in texture.

We do not use LPIPS, a distance computed from ImageNet image features that has no meaning for brightness temperatures. Instead, for a scene $x$ and pixel $p$, the probe score is

$$ s(p) = \operatorname{median}_{k=1..K} \sum_c w_c \, \big| x_c(p) - \hat{x}^{(k)}_c(p) \big|, $$

where $\hat{x}^{(k)}$ is the $k$-th of $K$ stochastic inpaintings under a different random mask, $c$ indexes channels, and $w_c$ rescales each channel by its training-set standard deviation. Taking the median over runs follows LMD.

We build two versions. The first is a diffusion inpainter, which is accurate but slow. The second is a single-pass masked autoencoder with its top skip connection removed. It is trained, as in DRÆM, alongside a small head that maps residuals straight to $s(p)$, so the operational latency question is answered rather than assumed away.

The virtual radar models whose errors we try to predict are a U-Net, an STR-UNet-style model trained on the land, coast and offshore mixture, and a DiffSR-style two-stage diffusion model. Using three synthesizers lets us test whether the probe is truly model-agnostic.

Evaluating a trust map for regions without ground truth requires an honest stand-in for those regions. We build pseudo-blind regions where radar does exist but is withheld from both the synthesizer and the probe. The cleanest case is the Himawari/CMA setting with Yu et al.'s land, coast, offshore and sea split. There we train on land and test on offshore and sea pixels that lie within range of coastal radars. In SEVIR and the MRMS/GOES CONUS data used by DiffSR, we hold out geographic blocks such as Gulf of Mexico coastal waters, the mountain West and the northern Plains, assuming event locations can be recovered from the dataset metadata. We also hold out a season and a time of day as non-geographic shifts. A second evaluation lives inside the radar mask: pixels near radar beam blockage or at the edge of radar range, where the radar "truth" itself is unreliable. The probe should not be rewarded for flagging these as errors, so we report them separately.

Baselines

The do-nothing reference assigns uniform confidence everywhere. Its risk-coverage curve, which plots error on the pixels kept against the fraction kept, is flat, and any trust map must beat it. A slightly smarter naive baseline is the geographic heuristic: confidence falls with distance from the training radar footprint, with a separate land/sea flag. This baseline is dangerous because in the Yu et al. split, distance from shore probably correlates strongly with error. We will report it on every split and require the probe to beat it within fixed distance bands, not only overall.

The strongest existing competitor is the synthesizer's own uncertainty. For the U-Net and STR-UNet models this is a five-member deep ensemble's per-pixel variance. For the DiffSR-style model it is the spread across eight diffusion samples. All three get the same training data, the same total GPU-hours as the probe, and the same hyperparameter search budget, which is tuned on an in-coverage validation split only, because tuning on the held-out region would leak the answer. The second strong competitor is feature distance: Mahalanobis distance of each patch's synthesizer encoder features from the training feature distribution. The third is a supervised error predictor, a network trained on in-coverage data to regress the synthesizer's absolute error from satellite input. A likelihood score from the diffusion inpainter itself is included for completeness, though LMD's 0.193 versus 0.985 result predicts it will do poorly.

For the synthesizers themselves, the strongest reported number in the file is DiffSR's CSI-50 of 0.277 against a UNet's 0.130 on MRMS composite reflectivity (DiffSR). CSI-50 is the Critical Success Index for pixels at or above 50 dBZ, the threshold for intense convection. We will reproduce DiffSR's in-coverage numbers before trusting our out-of-coverage comparisons. For the trust-map task itself, no reported result was found on Lacuna for predicting virtual-radar error in held-out geography. The baselines above therefore define the bar.

The ablations isolate the main ingredients. We compare spatial-only against channel-only against combined masks. We compare a probe with full skip connections against one without, to test DeScarGAN's warning. We compare diffusion against single-pass reconstruction.

Beating the strongest baseline means three things on the offshore/sea split and on at least two CONUS held-out blocks. First, higher ROC-AUC than ensemble or diffusion spread at flagging patches whose CSI-35 (the success index at 35 dBZ, a common threshold for convective rain) falls below the in-coverage median. Second, at 70% coverage, the CSI-35 on retained pixels must improve more when pixels are dropped by the probe than when they are dropped by ensemble spread. Third, a combined score must beat ensemble spread alone with confidence intervals from a block bootstrap over storm events that exclude zero.

Why not just…?

Why not just use ensemble disagreement or diffusion sample spread? This is the real threat, and it may well win when the synthesizer is the DiffSR-style model, whose samples already vary most in convective regions. The argument against relying on it alone is that ensemble members share training data and the same blind spots. Under a systematic input shift, such as Yu et al.'s change in water vapor reliance, the members can agree confidently on a wrong answer. The combined-score test settles this. If the probe adds no ROC-AUC beyond spread on the ocean split, we report the probe as redundant there and keep only the evaluation protocol as the contribution.

Why not just train a network to predict the synthesizer's error? Such a network learns which errors occur inside radar coverage, which is exactly where we do not need it. It has no reason to extrapolate correctly to inputs it never saw. We include it as a baseline. Our hypothesis is that it does well in-coverage and falls behind the probe on held-out regions. If it does not fall behind, it should replace the probe, because it is simpler.

Why not just train the synthesizer on more diverse data, as Yu et al. recommend? That improves accuracy and we adopt it: the STR-UNet-style synthesizer is trained on the land, coast and offshore mixture. But a better model still has failure regions, and diverse training does not show where they are. The two approaches answer different questions, so they complement each other.

Why not just flag by distance from radar or the land/sea mask? Because it is free and probably strong, it is a baseline on every split. It cannot react to the weather itself. Two scenes at the same offshore distance, one a familiar frontal band and one an unusual tropical system, get the same confidence. Comparing the probe and the heuristic within distance bands tests exactly this.

Decisive Checks

Three results decide the proposal. The first is whether the combined probe-plus-spread score beats spread alone on the Himawari offshore and sea split. Second, whether that gain survives within distance-from-coast bands. Third, whether one probe, trained once, helps all three synthesizers, which tests the model-agnostic claim. A null result on the first check, with the probe beating only the uniform and distance baselines, would still be informative. It would mean that at present a virtual radar model's own ensemble is the best available trust signal for blind regions, and that the pseudo-blind protocol, not the probe, is the reusable artifact.