Forest dieback from bark beetles is a slow disease of a canopy. A tree is first attacked while it still looks green, then yellows, reddens, and finally turns gray, and the outbreak spreads from infested trees to their neighbors. The labels foresters actually hold are coarse. They are stand-level summaries, such as "about 30% of this compartment was salvage-cut after 2018." Pixel-level damage masks are rare and expensive. We propose to learn, from those coarse labels alone, a per-pixel progression score. This score is a number between 0 and 1 that only increases over time and tracks how far each 10 m patch of forest has moved from healthy toward dead. We learn it from Sentinel-2 satellite time series using multiple instance learning (MIL): training where only groups of pixels, called "bags," carry labels, and the model must infer what happened to each pixel, called an "instance."
The idea has two ingredients that current work lacks, and each fixes a specific way this problem goes wrong. First, warming has lengthened and shifted the growing season. The greenness drop that signals beetle stress in one year can simply be early autumn senescence in another. We therefore measure time from each pixel's own green-up date rather than by the calendar. Second, we bring neighboring pixels in only through a spread term: a neighbor's damage raises a pixel's risk of being attacked. We do not feed neighbors in as extra image context. Remote sensing work shows that ignoring spatial structure hurts, while surgical imaging shows that models leaning on context collapse when geometry changes. The spread term is meant to capture the first benefit without inheriting the second failure.
Core Claim
A monotone, cumulative-hazard progression model can be learned from bag-level damaged fractions. Two conditions matter. Its clock must be the pixel's phenological time, and its spatial dependence must enter through neighbors' latent progression rather than neighbors' appearance. Under these conditions the model should approach fully supervised segmentation on pixel-level dieback. It should also reduce false alarms in years with anomalous season timing. And its intermediate scores should order trees by time to mortality, which a binary mask cannot do.
Where This Sits
Three bark beetle studies motivate the gap by solving adjacent problems under stronger supervision.
DIAMANTE segments dieback with a U-Net on fused Sentinel-1 radar and Sentinel-2 optical imagery. It needs full pixel masks and outputs a binary map. We keep its data and sensors but replace mask supervision with stand fractions, and replace the binary output with a continuous score.
Combining Sentinel-2 Data and Risk Maps classifies attacked versus healthy stands with pixel-wise random forests. It uses harvester coordinates and static risk maps, and treats neighborhoods only as a 3×3 coefficient of variation. We treat neighborhood as a dynamic spread process.
Kapil et al. detect green, yellow, red and gray crowns from drone imagery with box-level labels. It confirms that the process is naturally ordinal, but it treats the stages as separate classes.
On the MIL side, Kvande et al. is the closest method. It treats farms as bags and parcels as instances, with an attention-based MIL model over Sentinel-2 time series. Its instances are unordered and its targets are crop categories or yield. We differ in that our target is a continuous, irreversible instance state, and our instances are spatially coupled. Topologically-regularized MIL adds structure among instances, but for blood cells rather than for spatial spread over a landscape.
The progression machinery is borrowed from medical imaging. DFPL factors future disease into current status plus the probability of progressing, which makes predictions irreversible by construction. The RMLP monotonicity constraint forbids spurious recovery. BrainODE models trajectories in continuous time so that irregular observation dates are harmless, which matters here because cloudy days leave gaps. All three assume subject-level longitudinal labels; we supply only bag-level ones.
Two papers set the spatial design. Li et al. show that ignoring spatial autocorrelation hurts: leaf area index error falls from 0.27 to 0.21 m²/m² when kriging, a spatial interpolation of residual errors, is added to spectral regression. Sellner et al. constrain the design from the other side. Segmentation models that learn organs from their surroundings lose about 45% of Dice score under geometric shift. We want the Li-style benefit without the Sellner-style failure.
The phenology motivation comes from Yan et al., who find green-up advancing about 15 days over 18 years. They also find that fixed-calendar rules misalign with actual vegetation in over 98% of their study area. A calendar-clocked damage detector inherits the same misalignment.
Model
Each instance $i$ is a 10 m pixel with an irregular Sentinel-2 time series (and Sentinel-1 where available). Following the harmonic-smoothing and water-index approach of Yan et al., we estimate that pixel's green-up date $g_i$ each year. The phenological clock is then $\tau = t - g_i$. As a variant, we accumulate growing degree days instead.
The progression score is a cumulative hazard, which is non-decreasing by construction:
$$s_i(\tau) = 1 - \exp\!\Big(-\int_0^{\tau} \lambda_i(u)\,du\Big), \qquad \lambda_i(u) = \mathrm{softplus}\big(f_\theta(x_i(u)) + \beta \textstyle\sum_{j \in N(i)} w_{ij}\, s_j(u-\Delta)\big).$$
The terms are defined as follows. $f_\theta$ is a small spectral-temporal encoder applied to the pixel's own observations $x_i$. $N(i)$ is the set of spatial neighbors, with distance weights $w_{ij}$. $\beta \ge 0$ is a bounded spread coefficient, and $\Delta$ is a lag of about one beetle generation.
Reversible fluctuations, such as drought stress that later recovers or cloud residue, are absorbed by a separate per-date nuisance head. That head feeds reconstruction but not $s_i$. This addresses the known tension between transient noise and monotone assumptions.
The bag loss matches the mean final score over a stand to the reported damaged fraction $y_B$:
$$\mathcal{L} = \sum_B \Big(y_B - \tfrac{1}{|B|}\textstyle\sum_{i\in B} s_i(T)\Big)^2 + \gamma\,\mathcal{L}_{\text{recon}}.$$
Here $T$ is the survey date, $\gamma$ weights an auxiliary reconstruction term, and $|B|$ is the number of pixels in the stand. We aggregate with a mean rather than a max because the target is a proportion. The one-positive-makes-positive rule of standard MIL is wrong for a gradual process.
The spatial term can raise a pixel's hazard but cannot by itself make the pixel look damaged to the observation model. Its influence is capped by a bound on $\beta$. That cap is the design choice aimed at the context crutch.
Research Program
The work has three stages, each tied to one of the claims.
The first stage builds weak-label versions of existing benchmarks, which lets us check pixel-level truth. On DIAMANTE, we aggregate the 2018 masks into stand-sized grid cells (we sweep 1, 5 and 25 ha) and keep only each cell's damaged fraction for training. We then test pixel-wise on the held-out 2020 temporal-transfer scenes. On the European spruce data, we aggregate harvester points into compartment-level fractions. The known harvest dates then serve as a continuous test: pixels harvested earlier should carry higher scores at a fixed phenological time.
The second stage analyzes the phenology question directly. We stratify test pixels by their green-up anomaly relative to their multi-year mean. We then measure false positives in late-season scenes for early-senescence and late-green-up strata separately.
The third stage probes the context crutch, borrowing Sellner's transplantation logic. We relocate clusters of healthy pixels next to dead patches, and dead clusters into healthy forest. We then measure how much each model's score shifts at the transplanted pixels. If the drone data of Kapil et al. are available, a secondary check treats crowns as instances and plots as bags. The test is whether the learned score recovers the green, yellow, red, gray order.
Baselines
The naive reference point is a calendar-anchored anomaly detector with no learning. For each pixel it computes a z-score of a water-sensitive index (shortwave-infrared or red-edge, as recommended by the early-detection review) against that pixel's own multi-year climatology on the same calendar dates. It thresholds the result, with the threshold tuned on training-bag fractions. A second naive reference predicts every pixel's score as its bag's mean fraction. Any learned model must beat both on pixel F1 and on bag mean absolute error, the average absolute gap between predicted and true stand fractions.
The strongest existing method is DIAMANTE's Middle-Fusion SUM U-Net trained on full pixel masks. It reports F1 of 70.93% on damaged areas in the 2020 transfer test, and 64.65% with optical data alone (DIAMANTE). Because it uses strictly more supervision than we do, it is an upper reference rather than a target to beat. We will rerun it with its published settings on identical scenes. Success for our weakly supervised model means exceeding the optical-only 64.65% while using only stand fractions. Approaching 70.93% with radar added would be a strong result. The same U-Net trained with our proportion loss on bag labels is the direct weak-label competitor. Our model must beat it at matched parameter count and tuning budget.
The strongest MIL competitor is an attention-based MIL model with a time-distributed recurrent encoder, of the kind used by Kvande et al.. It has unordered pixels and per-pixel outputs read from a regression head. It gets the same inputs, the same bags and the same hyperparameter search budget (an equal number of trials on a shared validation split of stands).
On the spruce data, the reference is the random forest with risk map, which reports 83–91% accuracy on Tile A and 92–97% on Tile B (Risk Maps). That model is trained on point labels, so matching it from compartment fractions counts as success. For the continuous score we found no reported result on Lacuna: nobody reports rank correlation with time to harvest. We therefore compare against the random forest's class probability used as a score.
Four ablations isolate the ingredients. The first uses a calendar clock instead of a phenological clock. The second removes the monotone hazard in favor of a free per-date sigmoid. The third sets $\beta=0$. The fourth replaces the spread coupling with a 5×5 feature-context CNN encoder, which gives spatial context through appearance.
Why not just…?
Why not just feed the green-up date to a calendar model as an extra input? This is the most serious threat, and we cannot rule it out in advance. A flexible encoder given $g_i$ could learn the time shift itself. The deciding experiment is the anomaly-stratified false-positive test on seasons more extreme than those seen in training. There a covariate must extrapolate, while an aligned clock does not have to. If the covariate variant matches alignment on those strata, we will report alignment as unnecessary and keep the simpler model.
Why not run standard attention MIL and smooth its output afterward with kriging or a conditional random field? Post-hoc smoothing is what Li et al. show works for static biophysical maps. It can reduce salt-and-pepper noise. But it spreads current damage symmetrically in space and has no notion of time lag. It therefore cannot express that damage now in a neighbor predicts damage later here, which is exactly the early-attack signal we care about. We include this as a baseline. The deciding measure is lead time: the number of weeks before red-attack that a pixel's score crosses a threshold fixed on training data.
Why not simply give the encoder a larger spatial patch? That is the route to the context crutch documented by Sellner et al.. It may well raise in-distribution F1. The transplantation test will show whether it pays for that gain with large score shifts when surroundings change. If it does not, the coupling term has no advantage and we will say so.
Why not collect a few pixel masks and fine-tune a supervised model? Where masks exist, that is the right call. Our mask-size sweep reports how many labeled hectares a supervised U-Net needs before it overtakes the bag-supervised model. That number is the practical answer.
Decisive Checks
The idea holds if three things are true on the DIAMANTE 2020 transfer set and the spruce tiles. First, our model beats every bag-supervised baseline on pixel F1 and on bag error. Second, removing phenological alignment raises false positives mainly in anomalous-season strata. Third, the monotone score has positive rank correlation with harvest date where the random forest's probability does not. The idea fails if a calendar model with a green-up covariate, followed by post-hoc kriging, matches it on all three. In that case, the contribution shrinks to a weak-label benchmark protocol, and we would publish it as such.
References
- DIAMANTE: A data-centric semantic segmentation approach to map tree dieback induced by bark beetle infestations via satellite images
- Combining Sentinel-2 Data and Risk Maps to Detect Trees Predisposed to and Attacked by European Spruce Bark Beetle
- Classification of Bark Beetle-Induced Forest Tree Mortality using Deep Learning
- Using AI to Empower Norwegian Agriculture: Attention-Based Multiple-Instance Learning Implementation
- Topological Inductive Bias fosters Multiple Instance Learning in Data-Scarce Scenarios
- Forecasting Irreversible Disease via Progression Learning
- BrainODE: Neural Shape Dynamics for Age- and Disease-aware Brain Trajectories
- Predicting Grassland Leaf Area Index in the Meadow Steppes of Northern China: A Comparative Study of Regression Approaches and Hybrid Geostatistical Methods
- Semantic Segmentation of Surgical Hyperspectral Images Under Geometric Domain Shifts
- Satellite-Based Monitoring on Green-Up Date for Optimizing the Rest-Grazing Period in Xilin Gol Grassland
- Early Detection of Bark Beetle Attack Using Remote Sensing and Machine Learning: A Review