Weather-Confounded Logs: Evaluating Stratospheric Relays the Way Epidemiologists Evaluate Treatments

Stratospheric relays are balloons, airships or solar aircraft parked about 20 km up. They let laser links between the ground and satellites avoid most of the turbulent, cloudy lower atmosphere. The case for them rests almost entirely on channel models and simulation. The first operators will soon be able to look at real logs instead, and those logs will carry a predictable trap. A sensible network sends traffic through the relay mainly when direct paths look bad: low satellite passes, forecast cloud, urgent data. A naive comparison of logged delivery rates will therefore mix the relay's benefit with the bad conditions that triggered its use. It could make a helpful relay look useless or harmful. Clinical epidemiology has a name for this, "confounding by indication": sicker patients get more aggressive treatment, so treatment looks worse than it is. Epidemiology also has a toolkit for studying treatments from records instead of trials, the retrospective cohort study.

We propose to build and validate that toolkit for relay networks before real logs arrive. We will generate logs from simulations driven by real historical weather, where the true effect of each routing choice is known. We will then test which retrospective-cohort estimators recover that effect. The estimators are propensity weighting, doubly robust estimation and survival models of delivery delay. We will also map where they break: forecast-versus-actual cloud that goes unrecorded, the network version of immortal-time bias, and shared onboard buffers that make one satellite pass's routing affect another's outcome. The deliverable is a validated analysis plan and a public benchmark for operational HAPS logs.

Core Claim

A pre-registered "target trial" design for relay logs can recover the causal benefit of stratospheric routing from confounded operational records. In this design, each satellite pass over a ground station is a patient, relay routing is the treatment, the start of the pass is the index date, and on-time delivery or time-to-delivery is the outcome. The naive log comparison cannot. We expect naive estimates to be biased toward zero, and possibly to reverse sign, under weather-aware routing. We expect the bias to be removable when the routing decision's inputs are logged. When they are not, or when passes compete for the same relay buffer, the bias should persist, and it can be bounded by sensitivity analysis.

The claim is useful only if logs from real HAPS operations eventually exist. The file notes that no study uses such logs today. We state this dependency up front rather than hide it.

Position Relative to Prior Work

The relaying literature supplies the physical motivation and the simulator. Integrated ground-air-space FSO links derives outage probabilities analytically and reports up to 30 dB outage reduction from relaying. SAG-FSO capacity analysis shows direct-link capacity falling from 3.8 to 1.7 Gbps between 30° and 80° zenith while the relayed path stays flat. Joint user association and beamforming optimizes a satellite-HAPS-ground network under perfect channel knowledge. All three are forward models and are silent on how to read observational data.

The closest work is the HAPS-enabled satellite IoT study. It uses five years of hourly Ottawa and Calgary weather, but compresses it into mean time-to-failure and time-to-recover figures that feed DtnSim, a simulator for delay-tolerant networks, which are networks that store data and forward it later when a link opens. It reports 645 relay contacts averaging 10 minutes against 518 direct contacts averaging 8 minutes, with near-100% delivery. We reuse its weather sources and simulator stack, but differ in three ways. We replay the actual hourly sequences, so storms keep their real persistence. We introduce a non-random logging policy. We treat the output as observational data to be analyzed, not as the answer.

The clinical papers supply the method and the cautionary examples. Tewari et al. use propensity scores to compare prostate cancer treatments assigned non-randomly. That is the core adjustment we port. Sternlicht et al. model the timing of an intervention as a per-day hazard in a setting where randomizing delay would be unethical. This mirrors our situation: operators will not randomize critical traffic during storms, and "how long to wait for the relay window" is a timing exposure. Chen et al. show real-world treatment choices driven by factors other than the official indication. This warns us that operator routing may depend on unlogged factors. The stratospheric ozone causal study applies causal discovery to atmospheric chemistry, not to communication decisions, and does not overlap.

Research Program

We will build a semi-synthetic benchmark whose ground truth is exact. We start from the 2017–2022 hourly MeteoBlue/Environment and Climate Change Canada archives used by Fettes et al.. We then generate LEO passes over Ottawa and Calgary with the Matlab Satellite Communication Toolbox. For each pass we simulate both arms under the same weather trace and the same random seeds: dump data to the HAPS and forward it later, or use the direct optical link. The per-pass difference is the true individual effect. Averaging it gives the true average effect on the delivery ratio and on delivery-time hazards.

Link quality within each arm follows the published channel models. These are Gamma-Gamma and exponentiated-Weibull turbulence, Hoyt-distributed pointing jitter for the platform, and Beer-Lambert cloud loss, all from Ata and Alouini. We add a simple solar-battery state for the platform so that relay availability varies by season and time of day.

A logging policy then chooses one arm per pass, and only that arm's outcome enters the "log". The policy uses variables available at decision time: forecast cloud cover, maximum pass elevation, traffic priority, battery state and buffer occupancy. We vary its strength from uniformly random routing, which serves as the randomized-trial reference, to strongly weather-driven routing.

Three failure modes are built in deliberately.

The first is hidden confounding. Actual cloud differs from forecast cloud and affects outcomes, and some policy variants log only the forecast.

The second is the network version of immortal-time bias. If exposure is defined as "was ever buffered on the HAPS," only packets that lived long enough to reach a relay window count as exposed. We contrast that definition with exposure fixed at the start of the pass.

The third is interference, meaning one unit's treatment changes another unit's outcome. Passes routed to the relay fill a shared buffer that later passes must compete for. This violates the usual assumption that units do not affect each other, the network-interference setting noted in the file.

Analysis follows cohort practice. We define the emulated trial before touching outcomes. We check covariate balance after weighting. We estimate delivery-ratio differences by weighting and doubly robust methods, and time-to-delivery with Kaplan-Meier curves and Cox models, censoring packets at their deadline. We handle interference by redefining units as relay contact windows, so that all passes sharing a buffer form one cluster.

Baselines

The do-nothing reference is the naive log comparison: delivery ratio of relayed passes minus delivery ratio of direct passes, computed on the logged data. This is what an operator would report first. Showing its bias is half the point.

The next baseline is outcome regression. We fit a logistic or Cox model of delivery on routing plus the logged covariates, the standard multivariable adjustment in Chen et al. and Ding et al.. It uses the same covariates and the same data as every other estimator.

The strongest competitor is off-policy evaluation as practiced in bandits and reinforcement learning, i.e., inverse-propensity and doubly robust estimators applied to the routing policy. It targets the same quantity and is the method a networking researcher would reach for.

A second strong competitor is trace-driven simulation. Here we simply replay the weather through the channel model and read off the relay benefit without using logs at all. On the benchmark this is the answer by construction. That is exactly why it is the right foil for the real-data case discussed below.

The reported numbers to beat are the simulator ground truths. Ata and Alouini report outage falling from about $10^{-2}$ to $2.7\times10^{-3}$ for a ground-to-GEO link at 50° zenith. Fettes et al. report near-100% delivery with the relay. We found no reported result on Lacuna for any causal estimator applied to non-terrestrial network routing logs.

All estimators get the same propensity-model family and the same tuning budget. That budget is a fixed number of cross-validated gradient-boosted and logistic fits. A method "beats" another if it has lower bias and RMSE of the average effect across 200 replicate logs, and if its 95% intervals cover the truth at close to the nominal rate.

The ablation that isolates our contribution keeps the estimator fixed and changes only the cohort design. It compares index-at-pass-start against ever-buffered exposure definitions, and pass-level against contact-window units. This separates what the epidemiological design contributes from what the weighting arithmetic contributes.

Why not just…?

Why not just run randomized A/B routing on the deployed network? Where it is allowed, this is better than anything here, and our uniform-policy arm plays exactly that role. But operators have the same reason clinicians do not randomize delayed intubation (Sternlicht et al.). Deliberately sending urgent data over a direct link into a storm costs real service. At best, randomization will cover low-stakes traffic in fair weather, which is the regime where the relay matters least. The benchmark will measure how much a small randomized slice, combined with the observational bulk, tightens estimates.

Why not just trust trace-driven simulation and skip causal inference? On the benchmark, simulation is exact, so this proposal adds nothing there. Its value appears only with real logs, where the thing in doubt is the channel model itself. Lognormal models fail beyond 62° zenith (Ata and Alouini), fading across hops is assumed independent (Samy et al.), and channel knowledge is assumed perfect (Liu et al.). A log-based estimate that disagrees with simulation is how those assumptions get tested. We concede that the benchmark validates the estimators, not the physics.

Isn't this just off-policy evaluation under a new name? The weighting arithmetic is shared, and we will say so. The difference is in the design choices that off-policy evaluation usually takes as given: fixing the index time to avoid immortal-time bias, choosing the unit to absorb buffer interference, handling censored delivery times, and bounding hidden forecast error with sensitivity analysis. If the design ablation shows these choices change nothing, the contribution shrinks to a benchmark, and we will report that.

Decisive Checks

The first check is the bias of the naive estimator. We expect naive log comparisons under weather-driven routing to understate the relay's true delivery benefit substantially, and under strong confounding to show no benefit or harm. If they do not, the confounding story is wrong for this system.

The second check is recovery with logged confounders. We expect weighting and doubly robust estimators with pass-start exposure to have near-nominal coverage when all decision inputs are logged.

The third check is recovery without them. When only forecast cloud is logged, we expect residual bias that grows with forecast error. The sensitivity analysis should bracket the truth. If it does not, the analysis plan needs hidden-variable measurement, such as sky cameras at the ground station, before any real study.

The fourth check is interference. Under high buffer load, pass-level estimators should fail and contact-window estimators should not. This check is the proposal's clearest claim that relay networks need more than an off-the-shelf clinical recipe.