Slow Canopies, Fast Soils: Closing the Vegetation Side Channel in Hybrid Radar Soil Moisture Retrieval

Radar satellites can see soil moisture through crops, but the canopy both absorbs and re-scatters the signal. What reaches the sensor is a mixture of a soil part and a vegetation part, and any retrieval method has to split the two. Recent hybrid models solve this by placing a physics equation inside a neural network. The network outputs a few physical quantities: soil moisture, surface roughness, and canopy parameters. A fixed, differentiable scattering model then turns those quantities back into the radar signal. The idea is attractive, but it has a hidden weakness. When labelled soil moisture is scarce, the network can use the canopy parameters as a free dial. It sets them to whatever values let it fit each training label and still reproduce the radar signal. The canopy variables then stop describing vegetation and start memorizing the training set, and the model fails once crops, years, or sites change.

We propose to treat this failure as a capacity problem, and to make the capacity limit physical rather than generic. Vegetation changes slowly over a season, while soil moisture can jump within a day after rain or irrigation. If the canopy latent of a hybrid model is forced to follow a smooth seasonal trajectory with only a few degrees of freedom, it can no longer absorb date-specific soil moisture residuals. Soil moisture then has to explain the fast part of the radar signal. Our claim is that limiting capacity on this one channel restores transfer across years and crop types. We expect it to do so better than regularizing the whole network with equal strength, and we expect it to make the canopy variables meaningful enough that an analyst can correct them by hand.

Core Claim

The closest existing work is the physical polarimetric autoencoder of Explainable Physical PolSAR Autoencoders for Soil Moisture Estimation. That paper trains an encoder whose outputs are explicit physical parameters. It decodes them with fixed surface, double-bounce and volume scattering models, and it uses reconstruction error as a physics-based reliability flag. It works pixel by pixel on single airborne L-band dates and puts no capacity control on its latent variables. In-distribution it is slightly worse than a plain supervised network: RMSE 7.11% versus 6.37% on CROPEX 2014. On the out-of-distribution HTERRA 2022 campaign it is reported only qualitatively, as better tracking where the black box collapses toward the training mean. We do not simply add a temporal encoder (the "spatio-temporal physical bottleneck" gap) or bolt a variational bottleneck onto that encoder. We make a narrower and testable claim, in three parts.

First, the transfer failure of hybrid and black-box retrievals under canopy is driven by interpolation through the nuisance channel. The model fits training labels exactly by moving the vegetation latents.

Second, the right remedy is to restrict the effective dimension of that channel along the time axis, where the physics says vegetation is slow.

Third, the regularization result of Interpolation can hurt robust generalization even when there is no noise should carry over in a targeted form. That paper proves that interpolating linear models have strictly higher robust risk than ridge-regularized ones even without label noise. We predict that out-of-distribution error falls as canopy-channel capacity is reduced, until the canopy can no longer be represented. At that point the physics reconstruction error rises sharply, which gives a label-free stopping signal.

For Sentinel-1 C-band data we use the Water Cloud Model, a semi-empirical canopy model already used in modified form by Ensemble Machine-Learning-Based Framework for Estimating Surface Soil Moisture Using Sentinel-1/2 Data, as the fixed decoder:

$$\sigma^0_t = A\,v_t\cos\theta\,(1-\gamma_t^2) + \gamma_t^2\,\sigma^0_{\text{soil}}(m_t),\qquad \gamma_t^2=\exp(-2Bv_t/\cos\theta).$$

Here $\sigma^0_t$ is backscatter on date $t$ and $\theta$ the incidence angle. $v_t$ is the canopy descriptor, $\gamma_t^2$ the two-way canopy transmissivity, and $\sigma^0_{\text{soil}}(m_t)$ the bare-soil response to soil moisture $m_t$. $A$ and $B$ are per-crop constants. The encoder outputs $m_t$ freely on every date. It is only allowed to output the canopy trajectory through a small seasonal basis:

$$v_t=\sum_{k=1}^{K} a_k\,\phi_k(t).$$

The $\phi_k$ are smooth B-spline functions spanning the growing season, the coefficients $a_k$ are predicted once per field-season, and $K$ is the capacity knob on the canopy channel.

Research Program

The work has three components.

Leakage diagnostics. We first build tools to measure leakage in existing hybrid models, using the released code of Basargin et al. on CROPEX 2014 and HTERRA 2022. We train a probe from the vegetation latents to the soil moisture residual left after the decoder fit, and measure it on held-out labels. If the probe recovers that residual, the canopy channel is carrying soil information. We also compare training error against the disagreement between co-located in-situ probes, which we measure from the networks themselves. Training error far below that disagreement is a sign of memorization. These tools apply equally to black boxes. The systematic bias Liu et al. report, overestimating above 10% and underestimating below 5% with $n=89$ labels, is the kind of shrinkage we expect from a model that memorizes in-sample and regresses to the mean elsewhere.

Temporally constrained hybrid retrieval. We then build the hybrid model for Sentinel-1 VV/VH time series, with Sentinel-2 indices as optional extra encoder inputs. We sweep $K$ from very flexible, with one canopy value per acquisition (which reproduces the unconstrained setting), down to two or three seasonal coefficients. The timescale split mirrors classical change-detection retrieval, and it is consistent with the finding of Seasonal variations in vegetation water content retrieved from microwave remote sensing over Amazon intact forests that static canopy assumptions produce contradictory seasonal artifacts. The canopy must vary in time, but in a structured way. We keep Basargin et al.'s reconstruction error as an abstention signal for saturated canopies. This matters because An assessment of L-band surface soil moisture products from SMOS and SMAP in the tropical areas shows that dense biomass defeats even L-band models, and no constraint can recover soil moisture there.

Interventions on the canopy channel. Finally, we test whether the constrained canopy latent is a real physical concept. Concept Bottleneck Models showed that correcting a few intermediate concepts at test time reduces task error. Are Concept Bottleneck Models Effective as Decision-Support Systems? found that this helps only when the concepts are objectively defined. Vegetation optical depth is objective, and it can be measured independently from Sentinel-1 itself following First Vegetation Optical Depth Mapping from Sentinel-1 C-band SAR Data over Crop Fields. We therefore overwrite $v_t$ with this external estimate, or with an optical-index proxy, and measure how soil moisture error changes. Earlier concept-model work used discrete visual attributes, so this is the first test we know of with interventions on a continuous canopy variable.

Baselines

Naive reference. The simplest predictor outputs the training-domain mean soil moisture, or per-station climatology on in-situ networks. It is the right floor because Basargin et al. report that their supervised network drifts toward exactly this behaviour out of distribution. Any method that does not clearly beat it on the target domain has not learned to transfer.

Classical inversion. We include a Water Cloud Model inversion with the canopy term driven by a Sentinel-2 vegetation index, plus a short-term change-detection retrieval. Both are fitted on the source domain only. This is the transparent, nearly parameter-free competitor. If it matches our model, the neural encoder is unnecessary.

Strongest black boxes. For pointwise retrieval we use CatBoost on the Liu et al. feature set. They report validation $R^2 = 0.827$ and RMSE 1.355% on the Weiku Oasis data, and stacking raises $R^2$ to 0.843. For time series we use InceptionTime, which Artificial Intelligence Algorithms for Rapeseed Fields Mapping Using Sentinel-1 Time Series shows is the most transfer-robust Sentinel-1 sequence model. It reaches a mean F1 of 92.7% under temporal transfer, while Random Forest collapsed below 5% F1 under a 30-day phenological shift. Following Donhauser et al., every black box gets an explicit tuning search over weight decay and early stopping. This keeps us from beating an under-regularized straw man.

Strongest hybrid. The physical autoencoder of Basargin et al. is the strongest hybrid, with reported in-distribution RMSE 7.11% on CROPEX 2014. Its supervised counterpart reaches 6.37%. No reported result was found on Lacuna for HTERRA 2022 RMSE, so we rerun their released code and report it ourselves.

Ablations. Three ablations isolate the key ingredient:

Fair comparison. All methods see identical inputs and labels. They receive the same number of hyperparameter trials and the same training compute. Model selection uses only a source-domain validation split; target-domain labels are never touched.

What counts as beating a baseline. Our method beats the strongest baseline if it achieves lower target-domain RMSE than both the tuned black box and the Basargin autoencoder on a majority of held-out years and sites. Significance is judged by a paired bootstrap over stations or fields. In-domain RMSE may be at most modestly worse. Accepting that small in-domain cost in exchange for transfer is the trade the theory predicts.

Why not just…?

Why not just regularize the black box properly? This is the most serious threat, because Donhauser et al. say ridge penalties and early stopping are exactly what fixes harmful interpolation. We take it head on: every black box is tuned for regularization strength. The global-regularization ablation tests the same idea inside the hybrid model. If tuned global regularization matches the canopy-channel constraint out of distribution, the targeted claim fails and we will report that. What a black box cannot provide in any case is the reconstruction-error abstention signal or a correctable canopy variable.

Why not just use the Water Cloud Model or change detection directly? These methods already encode the slow-canopy, fast-soil intuition and have almost nothing to overfit. But they use fixed canopy descriptors from optical indices. Wang et al. show such static or proxy assumptions break across seasons, and Bazzi et al.'s threshold-based irrigation pipeline fell from 93.0% accuracy in dry 2019 to 72.8% in humid 2017 (An Operational Framework for Mapping Irrigated Areas at Plot Scale Using Sentinel-1 and Sentinel-2 Data). The classical baseline is run on every split. If it matches us, the learned encoder adds nothing.

Why not just add a temporal encoder to the physical autoencoder? That is the obvious extension, and it is our first ablation. Our prediction is that it makes leakage worse. A sequence encoder with a per-date canopy output gains capacity in precisely the channel that leaks. The leakage probe and the $K$ sweep will show whether this happens.

Why not just put an information bottleneck or Bayesian sparsity on the encoder? These methods compress whatever is cheapest to compress. They have no reason to remove soil information from the canopy channel in particular, and the file notes their hyperparameters are brittle and the Bayesian sampling is costly. The matched-capacity ablation tests whether physically placed capacity beats generically placed capacity.

Proposed Experiments

Experiment 1: leakage in an existing hybrid. We train Basargin et al.'s model on CROPEX 2014 with and without a capacity limit on its volume-scattering latents. Within a single date the limit is a width and penalty restriction, since there is no time axis. We test on HTERRA 2022. The decisive check is whether out-of-distribution RMSE tracks the leakage probe score across these variants.

Experiment 2: transfer across years and stations. This experiment uses Sentinel-1 series matched to the REMEDHUS and SCAN in-situ networks, the same networks used in Dynamic Structure Learning through Graph Neural Network for Forecasting Soil Moisture in Precision Agriculture. That paper does forecasting rather than retrieval and gives no scalar numbers we can compare against. We hold out whole years and whole stations, and plot target-domain RMSE and reconstruction error against $K$. The prediction is that out-of-distribution error improves as $K$ shrinks. It should bottom out where reconstruction error begins to climb. Applying the same constraint to the soil channel should give no such improvement. The main confound is the mismatch between point sensors and 10 m pixels. We report error against inter-probe disagreement so that this confound does not masquerade as model error.

Experiment 3: dry-to-wet transfer for irrigation. On the French plot-scale Sentinel-1 series from Bazzi et al., we use retrieved soil moisture jumps to detect irrigation. We train in dry years and test in humid 2017, where the reported S²IM accuracy is 72.8%. The InceptionTime and Random Forest baselines are run on identical splits.

Experiment 4: canopy interventions. We replace $v_t$ with independent Sentinel-1 vegetation optical depth or optical proxies at test time. The decisive check is a sign difference. Intervention should reduce soil moisture error for the constrained model and fail to help, or actively hurt, the unconstrained one. Seeing that pattern would show that only the constrained canopy latent means vegetation.