Weakly supervised land-cover mapping trains a model to label every small tile of a satellite image while only ever being told the scene-level answer, for example "40% forest, 35% farmland, 25% water." The leading approach treats the image as a bag of tiles. Each tile gets a class prediction, the predictions are averaged, and the average is matched to the scene percentages. This setup is called multiple instance learning (MIL): labels attach to sets of items rather than to the items themselves. The averaging step cannot see where tiles sit. A checkerboard of forest and farmland and two solid blocks of the same proportions receive identical loss. Landscape ecologists, however, care most about this difference. Their fragmentation measures, such as contagion (how often neighboring cells share a class), edge density, and patch cohesion, exist to separate clumped landscapes from shattered ones. A map that gets proportions right and arrangement wrong gives wrong fragmentation estimates.
We propose to add one cheap, second-order weak label to the first-order proportions: the scene's class adjacency matrix, meaning how often each pair of classes appears side by side. Under soft tile predictions, the adjacency matrix has a closed-form differentiable expectation. It determines contagion and edge density exactly, and it constrains arrangement without saying which tile is which. The core question is whether configuration labels improve tile-level maps and landscape-metric fidelity beyond what an equal number of extra proportion labels would buy. A second question is whether they beat the isotropic smoothness priors that medical MIL uses to restore spatial context.
Core Claim
Spatial context loss in MIL is usually treated as an architecture problem, and the fixes are architectural: graphs, sparse convolutions, distance-decayed attention. We argue that in proportion-supervised Earth observation it is first a loss problem. A proportion loss is exactly invariant to any rearrangement of tile predictions. A spatially aware feature extractor can only break that invariance indirectly, through whatever correlations happen to exist in the features. Supervising the expected adjacency matrix breaks it directly. It also does so without assuming that neighbors should agree, which matters because fragmented landscapes are precisely the ones where neighbors disagree.
Concretely, let $p_i \in \Delta^{K}$ be the predicted class distribution of tile $i$ over $K$ classes, and let $E$ be the set of 4-neighbor tile pairs on the grid. The predicted adjacency matrix is
$$\hat{A} = \frac{1}{2|E|}\sum_{(i,j)\in E}\left(p_i p_j^\top + p_j p_i^\top\right),$$
a symmetric $K\times K$ matrix whose diagonal is same-class adjacency and whose off-diagonal mass is edge density. The training objective is
$$\mathcal{L} = \lVert \bar{p} - y \rVert^2 + \lambda \lVert \hat{A} - A \rVert_F^2,$$
where $\bar{p}$ is the mean tile prediction, $y$ the scene proportions, $A$ the target adjacency matrix computed the same way from per-cell class fractions at the same grid resolution, and $\lambda$ a weight. Computing target and prediction at the same grid scale sidesteps the known resolution sensitivity of landscape metrics: we never compare a pixel-scale metric to a tile-scale one. Contagion and edge density are functions of $(\bar{p}, \hat{A})$, so fitting this loss is fitting those metrics.
This differs sharply from the isotropic smoothness priors in ProbSA, which penalizes attention differences between adjacent tiles with a Dirichlet energy term, and in SMMILe, which adds a Markov random field penalty on label disagreement between neighbors. Both push every scene toward high contagion. Our loss pushes each scene toward its own contagion, which may be low.
Relation to Prior Work
Two papers define the regime and motivate the gap. Scene-to-Patch Earth Observation introduced proportion-supervised instance-level MIL on DeepGlobe, averaging tile predictions on an 8×8 grid. Its aggregation is orderless by construction. Multi-resolution MIL for Earth observation diagnosed "geographically incoherent predictions" from that orderlessness and added multi-scale grids (8×8 to 32×32) whose coarse embeddings are repeated onto fine tiles (MRMO). It left horizontal spatial interaction and landscape metrics to future work, and its loss remains proportion-only. We keep its architecture and change the supervision.
SMMILe supplies a methodological result we rely on. In pathology, instance-level aggregation preserves spatial quantification better than representation-level attention, which tends to latch onto a few discriminative tiles. This justifies building on per-tile predictions, which is also what makes $\hat{A}$ computable.
Several pathology methods restore context architecturally: SparseConvMIL and SparseXMIL (sparse convolutions over tile coordinates), GDS-MIL (graph attention over 8-neighbors, which oversmoothed when deepened), and PSA-MIL (distance-decayed attention). These are complementary to our proposal rather than competing with it, and we test the combination. Their reported gains are for slide classification, not for tile maps under proportion labels. The oversmoothing in GDS-MIL is a warning about any mechanism that averages neighbors in fragmented scenes.
The landscape-ecology papers motivate the evaluation. Studies tracking fragmentation via contagion, Shannon diversity and cohesion in Jiangsu (Shi et al.) and via patch counts in mangroves (Shirmohammadi et al.) compute these metrics on hard maps after the fact. The mangrove study saw patch counts move from 1,281 to 911 to 1,639 across wet and drought periods. In such work, the map's configuration error propagates directly into the scientific conclusion.
Research Program
The work has three parts. First, we build the adjacency-supervised variant of the MRMO model on its 32×32 output grid. For DeepGlobe's 7 classes, the symmetric adjacency matrix adds 28 numbers per scene. Targets are derived from the ground-truth masks to simulate a configuration label. A realistic source in practice is a coarse existing map, such as the 30 m CLCD product used in the Yellow River mining risk study. We treat that as a follow-up, because coarse-map adjacency has its own scale mismatch.
Second, we introduce landscape-metric fidelity as an evaluation axis for weakly supervised maps. On held-out scenes, we compute contagion, edge density, landscape shape index, number of patches, and largest patch index from predicted hard maps and from ground truth, and report the error. No reported result found on Lacuna for landscape-metric error of any weakly supervised MIL model.
Third, we analyze the regime: performance stratified by the true contagion of each test scene. The hypothesis is specific. Smoothness priors help clumped scenes and hurt fragmented ones, while adjacency supervision helps across the range.
Baselines
The naive reference is single-resolution Scene-to-Patch with mean aggregation and proportion loss. It reports 0.397 pixel mIoU and 0.090 proportion RMSE on DeepGlobe (S2P). It shows what orderless MIL achieves and gives the floor for metric fidelity.
The strongest existing method is MRMO from multi-resolution MIL. It reports 0.417 pixel mIoU on DeepGlobe and 0.271 on FloodNet, the best weakly supervised numbers in the file; for reference, a fully supervised DeepLab baseline in the DeepGlobe challenge paper reports 0.433. It is the right competitor because it already addresses spatial incoherence architecturally in exactly our setting. We will reproduce it on the same splits, backbone, grid sizes and epochs, with identical hyperparameter search budgets for it and for every variant. Our model is MRMO plus the adjacency term, so the only difference is the loss. Beating it means a mIoU gain beyond seed variance over at least five seeds on both datasets, together with a clear reduction in contagion and edge-density error.
The key ablation is an equal-information control: MRMO trained on proportion labels for each of the four scene quadrants, which is 28 numbers per scene, the same count as the adjacency matrix. If quadrant proportions match adjacency labels, the gain comes from extra label scalars and not from configuration.
The smoothness controls are MRMO plus a Dirichlet-energy penalty on tile predictions, following ProbSA, and MRMO plus an MRF disagreement penalty, following SMMILe. These need no extra labels and represent the standard way to inject spatial context. Each gets its weight tuned on the same budget.
Finally, an architectural control replaces the averaging head with a sparse-convolution head over tile coordinates, as in SparseConvMIL, trained with proportion loss only. We also run it with the adjacency term added.
Why not just…?
Why not just add a smoothness penalty, which needs no new labels? This is the most natural alternative, and on clumped scenes it may match our method. Dirichlet energy and MRF penalties, however, encode a single prior: neighbors agree. That prior is wrong for interspersed land cover, where low contagion and high edge complexity are the signal ecologists measure (Yu et al. use fractal dimension precisely to detect farmland broken into irregular shapes). The contagion-stratified analysis settles this. If smoothness priors match adjacency supervision in the lowest-contagion third of test scenes, our claim fails.
Why not just use a spatially aware architecture such as sparse convolutions or graph attention? Architecture lets tile features see neighbors, and that can help recognition. But the proportion loss stays invariant to rearranging predictions, so nothing in the objective rewards a correct arrangement. Pathology evidence for these architectures concerns slide-level accuracy, for example SparseConvMIL's 0.568 versus Attention-MIL's 0.487 on TCGA pan-cancer, not tile maps. GDS-MIL reports oversmoothing with depth. Our sparse-convolution control, run with and without the adjacency loss, shows whether architecture alone closes the gap. We expect the two to be complementary rather than substitutes.
Why not just use smaller bags, giving proportion labels for sub-regions? This is the real threat, because finer proportions also localize. The quadrant-proportion control uses exactly the same number of label scalars. If it matches adjacency supervision on mIoU and on metric fidelity, we will report that configuration labels offer no special value. Our hypothesis is the opposite: they win on fragmentation metrics, since quadrant proportions say nothing about arrangement within a quadrant.
Why not just collect a few dense masks and train a segmentation network? The file shows that a fully supervised UNet 448 reached only 0.290 mIoU on DeepGlobe (S2P), below the weakly supervised MIL models. Dense masks are also the costly annotation that weak supervision exists to avoid. We include it as a reference point rather than a competitor.
Proposed Experiments
We train all models on DeepGlobe land cover (803 scenes of 2448×2448 pixels, 7 classes, the S2P split) and on FloodNet, using five seeds each. We evaluate pixel mIoU, proportion RMSE, and landscape-metric errors (contagion, edge density, landscape shape index, number of patches, largest patch index), all computed on predicted versus true maps at the same grid resolution.
The first decisive check is the equal-information comparison. Adjacency supervision must beat quadrant-proportion supervision on contagion and edge-density error, and must at least match it on mIoU. The second check is the contagion-stratified comparison against the Dirichlet and MRF priors. The predicted signature is that smoothness priors improve clumped scenes but increase metric error on fragmented scenes, while adjacency supervision improves both. The third check is the 2×2 of averaging versus sparse-convolution head crossed with with versus without the adjacency loss, which separates architectural from supervisory context.
We also run a robustness check. We corrupt target adjacency matrices with noise and with coarse-resolution computation, simulating labels derived from a 30 m map, to measure how much configuration-label quality the method tolerates. Success means beating MRMO's 0.417 DeepGlobe and 0.271 FloodNet mIoU beyond seed variance, while substantially lowering fragmentation-metric error. The second criterion matters most to anyone using these maps for ecological risk work.
References
- Probabilistic smooth attention for deep multiple instance learning in medical imaging
- Accurate spatial quantification in computational pathology with multiple instance learning
- Scene-to-Patch Earth Observation: Multiple Instance Learning for Land Cover Classification
- Extending scene-to-patch models: Multi-resolution multiple instance learning for Earth observation
- SparseConvMIL: Sparse Convolutional Context-Aware Multiple Instance Learning for Whole Slide Image Classification
- SparseXMIL: Leveraging sparse convolutions for context-aware and memory-efficient classification of whole slide images in digital pathology
- Enhancing PFI Prediction with GDS-MIL: A Graph-Based Dual Stream MIL Approach
- PSA-MIL: A Probabilistic Spatial Attention-Based Multiple Instance Learning for Whole Slide Image Classification
- Spatiotemporal Dynamics and Prediction of Habitat Quality Based on Land Use and Cover Change in Jiangsu, China
- Assessing climatic change impacts on mangrove structural dynamics on the northern coasts of the Persian Gulf
- Assessing Landscape Ecological Risk from Mining in the River Source Region of the Yellow River Basin
- DeepGlobe 2018: A Challenge to Parse the Earth Through Satellite Images
- Decreased Landscape Ecological Security of Peri-Urban Cultivated Land Following Rapid Urbanization: An Impediment to Sustainable Agriculture