Planning Against Adaptive Stressors with Aggregate Threat Fields

Conservation models such as InVEST score each patch of land by adding up how much nearby roads, settlements, tourist sites and farms degrade it. Each stressor's influence fades with distance. The stressors themselves are fixed map layers. In reality many stressors are people: herders, tourists and harvesters who move, and who change where they go when a manager closes a pasture or a trail. A manager who plans closures against today's threat map will be surprised when the threats move. A manager who plans against the people themselves faces two problems at once. The number of possible joint movements is astronomically large, and the people keep adapting to the manager, so any fixed model of them goes stale. That second problem is what multi-agent learning calls the moving-target problem.

This note proposes a way through both problems that rests on one structural fact. Before the final saturation step, InVEST degradation is a linear sum of distance-decayed threat presence. Suppose we track only the probability that each cell hosts a stressor, rather than simulating individual agents. Then the expected degradation can be computed exactly by convolving that probability map with the decay kernel, with no independence error at all. A planner that propagates these probability maps forward, instead of sampling concrete trajectories, can therefore evaluate closure plans against hundreds of adapting agents cheaply. The only approximation error sits in the nonlinear saturation, and a variance term can correct it. We make the planner anticipate adaptation by putting the population's learning rule inside the look-ahead, not just its current behaviour.

Core Claim

Aggregate simulation means planning by pushing marginal probabilities through the dynamics rather than sampling concrete states. Factored MCTS for Large Scale Stochastic Planning introduced it (ARollout, and AMCTS, which combines a concrete tree near the root with aggregate rollouts at the leaves). It breaks the scalability wall of sampling planners, scaling to 264 state variables and 200 action variables. It fails when the objective depends on correlated variables, as in their Elevators domain. That failure is the constraint this proposal is built around.

Our claim is that spatial threat-decay objectives are a case where the independence assumption is nearly free. Let $p_{ry}$ be the probability that stressor type $r$ occupies cell $y$ at a given step. Write $K_{rxy} = \frac{w_r}{\sum_r w_r} i_{rxy} \beta_x S_{jr}$ for the InVEST weight that cell $y$'s threat contributes to cell $x$. This weight combines the threat weight $w_r$, the distance decay $i_{rxy}$, the protection level $\beta_x$ and the habitat sensitivity $S_{jr}$. By linearity of expectation, the mean and (under cell-wise independence) the variance of degradation are

$$\mu_x = \mathbb{E}[D_x] = \sum_{r,y} K_{rxy}\, p_{ry}, \qquad \sigma_x^2 \approx \sum_{r,y} K_{rxy}^2\, p_{ry}(1-p_{ry}).$$

The mean is exact regardless of correlations between agents. Only the variance uses independence. The habitat quality $Q_x = S_j\,(1 - f(D_x))$ applies a sigmoid saturation $f(D) = D^z/(D^z+k^z)$. We estimate expected quality with a second-order correction, $\mathbb{E}[Q_x] \approx S_j\,(1 - f(\mu_x) - \tfrac12 f''(\mu_x)\sigma_x^2)$, so the residual bias is third-order in the spread of $D_x$.

The claim is testable. The corrected aggregate estimate should track Monte Carlo ground truth closely wherever threat kernels are wide relative to agent clustering. It should degrade exactly where agents herd tightly, because there the positive correlation inflates the true variance beyond $\sigma_x^2$. This yields a predicted failure boundary rather than an unexplained one.

Research Program

Testbed. We build a spatial simulator on the Gannan Tibetan Autonomous Prefecture data used in Balancing tourism development and habitat conservation in fragile ecosystems: A case study of the Qinghai-Tibet Plateau. That study mapped tourism infrastructure (1,826 restaurants, 1,132 accommodations, 118 attractions) against 30 m land cover and found county-level habitat quality ranging from 0.82 to 0.31. We reuse its land cover, its OpenStreetMap roads, and its InVEST threat weights, reach distances and sensitivities. Those parameters are, as Dong, Huang et al. note, expert-set rather than calibrated. We treat them as given and run sensitivity sweeps rather than claiming ecological truth.

On this landscape we place a population of stressor agents: tourist cohorts that travel from accommodation points to attractions, and herders who seek high-suitability pasture. Each agent is a depth-limited planner with utility discounting. We use this form because Approximate planning in spatial search found that such models explain human spatial search far better than optimal expected utility (relative log-likelihood advantage 764; correlation with human choices 0.81 versus 0.49). Agents also learn across seasons: each updates softmax preferences over destinations from realized payoffs. The population is therefore non-stationary from the manager's point of view.

Manager. The manager acts at the township-zone level each season. It can close or permit zones, which raises $\beta_x$ protection and imposes access costs on agents. Its reward trades off expected habitat quality, converted to money with the locally adjusted equivalent-factor method of Yu et al., against tourism and grazing income. The action space is factored across zones, which gives the same combinatorial structure AMCTS was built for.

Planning mechanism. The planner keeps a concrete search tree over the manager's zone decisions for the first one or two seasons. At the leaves it runs aggregate rollouts. In these rollouts the population is a set of occupancy probability maps, one per stressor type. The maps advance each step through the agents' softmax movement rule, which is a differentiable function of access costs, and degradation is evaluated with the two equations above.

The moving-target component is that each rollout also advances the population's learning step, so agents' destination preferences in the rollout shift in response to the closures the manager chose. The planner therefore scores a closure by where people will go after they adapt, not by where they are now. The parameters of that learning rule are unknown: discount rate, sensitivity to access cost, learning rate. The manager holds a particle belief over them and updates it from observed occupancy each season.

Observed occupancy counts can be sharp, which risks collapsing the particles. We therefore reuse annealed importance resampling from Observation Adaptation via Annealed Importance Resampling for Partially Observable Markov Decision Processes, which reported a return of −12.93 on Laser Tag against −18.11 for POMCP. Here it is a supporting component, not a contribution.

Analysis. We will derive, and check empirically, the bias of the corrected estimator as a function of agent clustering. This gives a practical diagnostic: estimate the pairwise occupancy correlation from rollouts, and fall back to concrete sampling in zones where that correlation is high.

How this differs from nearby work. The closest bridges in the file each lack one ingredient. Optimising Spatial Teamwork Under Uncertainty plans spatial positioning against a continuous threat field with MCTS and a centralized solver, cutting opponent expected threat by 20.8%. However, its opponents are not learning, and its search is concrete. Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem trains a manager against cloned human behaviour, but in a non-spatial game with a scalar pool. Melting Pot's Clean Up (Melting Pot 2.0) couples activity to degradation through a step-function rule rather than distance decay, and evaluates only model-free learners. NonZero tames joint-action explosion by focusing on pairwise interactions. We avoid enumerating agents at all by moving to occupancy fields, which is only possible because the objective is kernel-linear.

Baselines

The do-nothing reference has two parts. One is no closures at all. The other is the standard practice of optimizing zone closures once, greedily, against the current static InVEST threat raster, and then holding them fixed. This second part is what a planner using existing habitat models would do. It is the reference that tells us whether modelling adaptation matters at all.

The strongest search competitor is concrete-sampling tree search over the same manager actions. Each rollout simulates every agent individually from the same population model and the same belief over its parameters. It is a POMCP-style planner with annealed particle resampling, so the only difference from our method is concrete versus aggregate leaf evaluation. Both get identical wall-clock time per decision; we sweep 1 to 60 seconds, matching the IPPC per-step limit used by Cui et al. Both get the same tuning budget for exploration constants.

On large IPPC SysAdmin instances, the reported result is that the PROST sampling planner "failed to complete its preprocessing phase within 5 hours," while aggregate methods ran within 60 seconds per step (Factored MCTS for Large Scale Stochastic Planning). No reported result was found on Lacuna for any planner on a spatial threat-decay agent simulator. Beating this baseline means higher seasonal habitat value at equal income, or equal value at lower compute, at population sizes of 100 or more agents. It must also not lose at small population sizes, where concrete sampling should be adequate.

The strongest learning competitor is a model-free manager trained offline by reinforcement learning against sampled populations from the same prior over learning-rule parameters. It is given the same total simulator steps that our planner consumes across all evaluation episodes. We use a policy-gradient manager, and for the zone-factored action space a factored critic in the style of FACMAC. This family is what the Koster et al. manager represents. That manager's reported headline in its own setting was sustaining 65% of games to the final round versus 5% under equal allocation. That number is from a different game and is not comparable here. Beating this competitor means better performance on held-out populations whose parameters lie outside the training prior.

Two ablations isolate the key ingredients. The first removes the population learning step from the rollouts, so the planner assumes agents keep their current preferences. This isolates the moving-target modelling. The second removes the variance correction, keeping only $f(\mu_x)$. This isolates the second-order term, and we measure its effect both on estimation bias against Monte Carlo ground truth and on decision quality.

Why not just…?

Why not just run concrete MCTS with more samples? When there are few agents, it may well suffice, and we expect it to. Our claim is about scaling. Cui et al. documented that concrete sampling becomes high-variance and eventually infeasible as state dimension grows. Tracking a few hundred agents on a 30 m grid is far past their 264 variables. The compute sweep in the baselines settles this by locating the crossover population size. If that crossover never appears below realistic population sizes, the proposal's case weakens to "cheaper, not better," and we would report that.

Why not just train a model-free manager offline against simulated populations? This is the real threat when the test population resembles the training population. A trained policy is cheap at deployment, and nothing in our method prevents it from working in-distribution. The evidence against it concerns shift. The Melting Pot evaluations (Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot) show that policies tuned to their training partners are exploited or break when partners change. We will test directly on held-out populations with shifted discount rates and cost sensitivity. If the offline manager matches our planner under shift, the online belief and look-ahead are not earning their cost.

Why not just re-run static InVEST optimization every season, reacting to observed threats? This greedy re-planning is a strong and simple heuristic, and it partially handles adaptation by following it. What it cannot do is anticipate displacement. Closing one pasture pushes herders into an adjacent, more sensitive one. Given the non-additive interaction between human pressure and fragile terrain that Yang et al. report, a reactive policy may pay that cost every season. We include seasonal greedy re-optimization as an additional baseline. The no-learning-step ablation measures how much of any gain comes from anticipation rather than from re-planning frequency. If agents barely adapt (low learning rates), we expect this baseline to tie, and the sweep over learning rate should show exactly where it stops tying.

Proposed Experiments

Implementation sanity check. First we reproduce aggregate-versus-concrete behaviour on IPPC SysAdmin and Elevators, to confirm our aggregate engine shows the published pattern: winning on the former, losing on the correlated latter.

Estimator bias. On the Gannan simulator we measure the bias of $\mathbb{E}[Q_x]$ estimates against 10,000-sample Monte Carlo ground truth, with and without the variance correction. We sweep agent clustering, from dispersed tourists to herds that move as blocks. The decisive check is whether bias stays small for dispersed populations and grows with measured occupancy correlation as the analysis predicts. If bias is large even for dispersed agents, the core claim is wrong.

Decision quality. We then compare all baselines on habitat quality averaged over a 10-season horizon, equivalent-factor ecosystem value, agent income, and compute per decision. We vary population size from 10 to 500, agent learning rate from zero upward, and the distance between training and held-out population parameters.

Robustness to expert parameters. We perturb threat weights and reach distances by ±50%. This checks that conclusions do not hinge on the uncalibrated expert settings that the habitat literature itself flags as unreliable.

What would settle the question. The proposal succeeds if two things hold together. First, the aggregate planner beats concrete search at equal compute once populations exceed a few dozen agents. Second, modelling the population's learning step beats both the stationary ablation and seasonal greedy re-planning whenever learning rates are non-trivial. It fails if either gain vanishes, and each failure has a specific diagnostic above.