Most environmental and health prediction models are built where the sensors are. The disparity is well documented. About 82% of randomized trials run in the richest fifth of countries (State of the evidence). Nearly 80% of regions studied in population-level machine learning for non-communicable diseases are high-income (Birdi et al.). Structured health transfer learning almost never moves toward low-resource regions (Bridging Data Gaps in Healthcare). Africa remains a "weak link" in carbon-cycle modeling because it has so few measurement stations (Africa's ecosystem productivity). These papers motivate the work; they do not supply a method. When a model trained on the United States is moved to Lima or Nairobi, the usual choice is to transfer everything and fine-tune, or to transfer nothing.
We propose a narrower bet. A quantity like ground-level PM2.5 at a place and time can be split into two parts. The first is a level: how polluted this place typically is, which depends on local emission sources and geography. The second is a response: how the value moves with its own recent history and with recent weather, for example the build-up after several stagnant days or a seasonal swing. We hypothesize that the response is largely shared physics and transfers across continents. The level is local and does not transfer. The design follows from this. A transformer, which is a sequence model that lets each time step look directly at any earlier step, learns the response from dense source regions and is then frozen. Only a small, location-conditioned level term is refit from the handful of target stations. This matters because ten stations cannot support fine-tuning a sequence model, but they can support estimating a few numbers per place.
Core Claim
Under severe target scarcity, freezing a source-trained temporal response model and refitting only a low-dimensional, location-conditioned offset and gain should outperform three alternatives. The first is full fine-tuning. The second is gradient-filtered joint training. The third is a static location embedding combined with seasonal time features. The advantage should shrink and eventually reverse as target station count grows, and it should vanish if the model is denied its covariate history. The paper is an empirical test of which part of a spatiotemporal model transfers. Neither the closest temporal-geographic work nor the closest transfer work separates the two parts, so neither can answer that question.
Model
Let $s$ be a location, $t$ a time, $c_{s,t-W:t}$ the trailing window of $W$ days of covariates at $s$, and $e(s)$ a fixed location embedding from SatCLIP. SatCLIP is a coordinate encoder pretrained on Sentinel-2 tiles sampled roughly uniformly over land, chosen precisely because it is less skewed toward rich regions. Let $\tau(t)$ be Fourier features of day-of-year. The prediction is
$$\hat y(s,t) = \mu_\phi\big(e(s),\tau(t)\big) + \big(1+\gamma_\phi(e(s))\big)\, g_\theta\big(c_{s,t-W:t},\tau(t)\big).$$
Here $g_\theta$ is a causal transformer over the covariate history; it is trained on the source and frozen. The function $\mu_\phi$ is a small level head, and $\gamma_\phi$ is a small scalar or low-rank gain head; both are refit on the target. The transformer only sees covariates available in both regions. On the US-to-Lima benchmark this means the 21 aligned features rather than the 77 US features. This choice avoids the source–target feature mismatch that Spatial Transfer Learning for PM2.5 handles with its Latent Dependency Factor. Missing days are handled by masked attention with explicit time encodings, which is the standard way to handle irregular sampling, as in TimeCHEAT.
The construction borrows parts from prior work. Fourier time features follow A Joint Space-Time Encoder, which fits a smooth field $f(x,y,t)$ with spherical harmonics and Fourier bases. Static location features come from SatCLIP. The new element is the split itself, plus the freeze-and-refit protocol it licenses.
Research Program
The work has three parts. First, a controlled data-desert benchmark built from the AI2 Climate Emulator subset used by the Space-Time Encoder paper. Because that emulator has ground truth everywhere, we can subsample "stations" in Africa and South America at realistically sparse densities: 2, 5, 10, and 30 stations per region. Stations elsewhere stay dense. We then test each method at held-out target locations and later times. Second, the real transfer case: US (1,081 stations) to Lima (10 stations) PM2.5, from the LDF paper. Here we use leave-one-station-out evaluation combined with a chronological split, so that both spatial and temporal generalization are tested. Third, a diagnostic analysis of the decomposition. We fit the full model on the target region alone, using the dense emulator, and measure how far the frozen source $g_\theta$ is from the target-optimal one. We do the same for $\mu$. Comparing these two distances directly tests whether the response transfers better than the level.
Baselines
The naive reference point uses no transfer. It is per-station climatology: a station mean plus a seasonal Fourier fit. Alongside it we run a target-only small transformer trained on the 10 Lima stations. If our method cannot beat seasonal climatology, nothing else matters.
The strongest existing method on the Lima benchmark is Nearest Neighbor Weighting with Latent Dependency Factors from Spatial Transfer Learning for PM2.5. That paper reports a 19.34% average improvement over baselines without the factor. No absolute MAE or RMSE for Lima was found on Lacuna, so we will rerun it on our splits rather than quote a number. It is the right competitor because it was built for exactly this pair of regions, but it uses static or daily-averaged features. We run it with the same 21 shared features, the same stations, and the same splits.
The strongest gradient-based transfer method is Target-Skewed Joint Training from Drawing Informative Gradients from Sources. It trains source and target jointly and discards the part of each source gradient that conflicts with the target gradient. It reports a 4.14% average gain over prior cross-city methods with 3 days of target data. We reimplement its gradient projection on our transformer backbone instead of Graph WaveNet, because Lima and the emulator regions have no road graph.
The field-fitting baseline is the Space-Time Encoder itself. On the emulator at 3% uniform density it reports RMSE 2.704 with Fourier time features and orthonormal regularization. That number comes from a uniform-density regime and is not comparable to our regionally sparse one, so we retrain the encoder under our station masks. For spatial interpolation we include SIGNN from Deep Spatial Domain Generalization, which reports MAE 9.40 against 12.44 for plain empirical risk minimization on Los Angeles PM2.5.
All transfer baselines get the same source data, the same covariates, and an equal hyperparameter search budget, tuned by inner leave-station-out validation on target data only.
Two ablations isolate the key ingredient. Setting $W=1$ removes the history and tests whether the dynamics are doing the work. The second ablation unfreezes $g_\theta$, which turns the method into fine-tuning. "Beating" a baseline means lower held-out MAE at 10 or fewer target stations, averaged over ten random station draws, with a paired test across draws.
Why not just…?
Why not just use SatCLIP embeddings plus seasonal time features in a regressor fine-tuned on the target? This is the most serious threat. If PM2.5 in Lima is mostly a static level times a seasonal cycle, this baseline captures it with fewer parameters. The Space-Time Encoder's gap between scalar and Fourier time features (3.144 versus 2.704 RMSE) shows that time encoding matters. It does not show that covariate history matters. The $W=1$ ablation together with this baseline settles the question: if neither beats our full model, we report that lagged response is not where the transferable signal lives.
Why not just fine-tune the whole source transformer on the target stations? With ten stations, fine-tuning has thousands of parameters per labeled location. TSJT reports that two-stage fine-tuning suffers negative transfer under few-shot targets, which is why it adds gradient filtering. Our station-count sweep on the emulator will show where fine-tuning catches up. We expect a crossover somewhere between 10 and 30 stations. If no crossover appears even at 2 stations, the freeze is unnecessary.
Why not just use gradient surgery as in TSJT, which already prevents negative transfer? Gradient filtering decides which source updates to accept, but it still adapts all parameters on target data. It also does not distinguish level from response. If TSJT matches us, the decomposition adds no value beyond regularization. The diagnostic distance analysis tells the two explanations apart.
Proposed Experiments and Decisive Checks
The headline experiment is held-out MAE versus target station count on the emulator's sparse African and South American masks, with every baseline above on the same curves. The second is the US-to-Lima result under leave-one-station-out evaluation.
The decisive check concerns the decomposition. On the emulator, the frozen source response must be measurably closer to the target-optimal response than the source level is to the target-optimal level. Our method must also lose its advantage when $W=1$. If the response is not more transferable than the level, the core hypothesis is wrong, and we would report that result. That negative would still be useful for anyone building models for the regions this literature documents as neglected.
A secondary check tests robustness across variables. We repeat the emulator study on temperature at several pressure levels, where the dynamics are smoother. If the method helps only for PM2.5-like quantities driven by episodes such as stagnant spells, we will say so.
References
- State of the evidence: a survey of global disparities in clinical trials
- Bias in machine learning applications to address non-communicable diseases at a population-level: a scoping review
- Bridging Data Gaps in Healthcare: A Scoping Review of Transfer Learning in Structured Data Analysis
- The interannual variability of Africa's ecosystem productivity: a multi-model analysis
- SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery
- Spatial Transfer Learning for Estimating PM2.5 in Data-Poor Regions
- TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysis
- A Joint Space-Time Encoder for Geographic Time-Series Data
- Drawing Informative Gradients from Sources: A One-stage Transfer Learning Framework for Cross-city Spatiotemporal Forecasting
- Deep Spatial Domain Generalization