Climate-Routed Adapters: Differentiable Trees That Decide Which Fine-Tuning Each Place Gets

Satellite foundation models are large networks pretrained on unlabeled imagery so that downstream tasks need only a few labels. They still fail when deployed on a continent they never saw. The usual remedy is to adapt one shared set of weights for everyone, but the relationships these models must learn, such as what flooded ground looks like on radar or how vegetation relates to biomass, differ between arid plateaus and wet lowlands. This note proposes a different remedy. A small decision tree, made trainable by gradient descent, reads a handful of physical descriptors of a location: climate, elevation, slope and land-cover fractions. Its leaves then pick which lightweight fine-tuning module the foundation model uses for that image. Because the tree splits on physical conditions rather than on latitude and longitude, a sample from an unseen continent is sent to the module trained on places with similar conditions elsewhere. That routing happens in one forward pass, with no extra training at test time.

In two sentences: replace the single low-rank adapter used in parameter-efficient fine-tuning with a shallow tree of adapters, routed by hard axis-aligned thresholds on environmental covariates and trained end to end with the backbone adapters. The claim is that this transfers better to held-out geographies than one shared adapter or than feeding the same covariates as extra inputs. The reason is that thresholds on physical variables are meaningful in places with no training labels, while coordinates and dense gates are not.

Core Claim

Two results in the file appear to conflict. Fine-tune Smarter, Not Harder compares LoRA (low-rank adaptation, which trains small rank-limited weight updates while the backbone stays frozen), prompt tuning, adapters and full fine-tuning on Prithvi 2.0 and Clay. It finds that appending coordinates and timestamps as metadata gives negligible gains. Meanwhile STAR splits a network into regional branches wherever prediction errors cluster in space, and raises California land-cover weighted F1 from 0.49 to 0.67. Foundation Models Meet Agriculture adds that vision encoders miss non-visual biophysical factors and that tabular models such as TabPFN often win.

Our reading is that location information helps when it decides which model handles a sample, and helps little when it is mixed into the features. It must also be expressed in variables that exist in the deployment region. STAR's partitions are tied to spatial units seen in training, so they cannot say anything about a held-out continent. Climate and terrain thresholds can.

What would be new is that the routing structure is a differentiable tree over physical covariates and that its leaves are adapters of a frozen foundation model, not output values. The claimed benefit is measured as transfer to held-out geographies at low label fractions.

Research Program

The model has three parts. The first is a frozen backbone, Prithvi 2.0 or Clay. The second is a shared LoRA update $\Delta W_0$. The third is a depth-$D$ tree with $D \in \{2,3\}$ whose internal nodes threshold single covariates $z_j$, and whose leaves $l$ each hold a small extra low-rank update $\Delta W_l = B_l A_l$. For an image with chip-level covariate vector $z$, each adapted layer uses

$$W(z) = W_0 + \Delta W_0 + \sum_{l} P(l \mid z)\, \Delta W_l ,$$

where $W_0$ is the pretrained weight and $P(l \mid z)$ is the probability that the tree routes $z$ to leaf $l$. The routing follows the straight-through recipe of GRANDE, which optimizes hard axis-aligned tree ensembles end to end. The forward pass is hard and one-hot, so inference costs the same as a single LoRA. The backward pass uses GRANDE's scaled-softsign surrogate, which keeps gradients alive for samples far from a threshold. Temperature annealing from RADDT, which anneals a softmin from soft to hard routing, is the fallback if training stalls.

Label efficiency enters in two ways. Leaf updates are given a smaller rank than the shared update and are penalized toward zero, so a leaf with few labels falls back to the shared adapter. Split thresholds are also initialized at quantiles of the covariates pooled over source and unlabeled target regions, which needs no labels.

Routing at the chip level, on coarse covariates, has a further advantage. It sidesteps the resolution-mismatch problem that Deploying an AI Application to Detect Flood from Sentinel 1 Data blamed for the marginal gains when 30 m elevation and 100 m land cover were concatenated with 10 m radar. The coarse variables never have to be resampled onto the image grid.

The analysis side asks three questions. Are the learned splits stable across random seeds? Do they align with known climate zones? How do target-region samples whose covariates fall outside the source range get routed? On that last question, an axis-aligned tree is guaranteed to send them to an edge leaf. A dense gate gives no comparable guarantee, and this is a testable property in our favor.

Baselines

The naive reference is a linear probe on frozen embeddings, which tells us how much any adaptation buys under shift.

The strongest existing method is a single shared LoRA, following Fine-tune Smarter, Not Harder. On the Sen1Floods11 geographic hold-out set it reports 87.57% mIoU for Prithvi 2.0 with LoRA, against 82.07% for full fine-tuning. We will match its rank to the total parameter count of our shared plus leaf adapters, run it in the same code and data splits, and give it the same hyperparameter search budget. We reproduce the 87.57% figure first. Beating it means exceeding the reproduced number by more than the seed-to-seed spread on the hold-out set, with no loss on in-distribution splits.

The covariate-as-input baseline uses the same LoRA but appends the same covariates to the decoder head's input. It isolates routing from mere access to the covariates.

The two-stage tabular baseline fits CatBoost and TabPFN on frozen embeddings concatenated with the covariates. TabPFN is the model that reached $R^2 = 0.65$ on US yield in Foundation Models Meet Agriculture, and we compare against that number on that task.

On MMEarth-Bench the comparison is TTT-MMR-GEO from MMEarth-Bench, which adapts the encoder at test time by reconstructing 12 modalities. No reported result was found on Lacuna for its per-task numbers, so we rerun it.

Four ablations isolate the key ingredient, and all keep the parameter count matched. The first replaces the tree with a dense multilayer gate over the same covariates, a soft mixture of adapters. The second uses a fixed partition with one adapter per region, built either from k-means on covariates or from climate-zone labels. The third is the same tree routed on latitude and longitude. The fourth is a tree with random, frozen splits.

Why not just…?

Why not just use one LoRA with a linear head? It is the strongest known approach on geographic hold-outs, and Foundation Models Meet Agriculture shows that one shared adaptation still produces model rankings that flip between countries. That flipping is what regime-specific adapters target. If the rank-matched shared LoRA ties us on the held-out continent, the idea has failed, and the experiments are set up to show this directly.

Why not just fit CatBoost or TabPFN on frozen embeddings plus covariates? This is a real threat. Gradient-boosted trees can split on climate too, train fast (CatBoost needed 41 s against NODE's 7.5 min in NODE), and win on several agricultural tasks. The tabular results favoring differentiable trees, such as GRANDE's mean reciprocal rank of 0.702 against CatBoost's 0.570, come from tabular data without spatial shift and do not settle the matter here. Our distinction is that the tree changes the backbone's features per regime instead of predicting from fixed features, which matters most for dense segmentation tasks like Sen1Floods11, where per-pixel boosting on embeddings is awkward. If the two-stage pipeline matches us on MMEarth-Bench regression, we will report that our contribution is confined to segmentation.

Why not just use fixed climate zones with one adapter each? This is the most dangerous alternative because it is cheap and interpretable. With few labels, a learned tree may simply rediscover climate zones. The fixed-partition ablation tests this head on. If it matches the learned tree, differentiability is unnecessary and we will say so.

Why not just do test-time training? TTT-MMR-GEO needs every auxiliary modality and gradient steps at deployment. Our method needs a few coarse covariates and one forward pass. The two can also be combined, which we test.

Proposed Experiments

We use the Sen1Floods11 geographic hold-out set (mIoU), the MMEarth-Bench Africa hold-out (biomass, soil nitrogen, soil organic carbon and pH, species occurrence), and the cross-country agricultural suite from Foundation Models Meet Agriculture. Each is run at 1%, 10% and 100% of source labels with five seeds.

The decisive checks are three. First, on held-out geographies, the routed adapters must beat both the rank-matched shared LoRA and the covariate-as-input LoRA at the 1% and 10% label fractions. Second, they must beat the dense gate and fixed-partition ablations most clearly on target samples whose covariates lie outside the source range, which is where the axis-aligned saturation argument predicts a gap. Third, the coordinate-routed tree should underperform the covariate-routed tree on the held-out continent. That result would confirm that what transfers is physical similarity, not position.

Interpretability is checked by printing the learned splits and by testing their stability across seeds. A gain that disappears under the fixed-partition ablation, or under CatBoost on segmentation-free tasks, will be reported as a negative result.