Playing games against nature: optimal policies for renewable resource allocation

Stefano Ermon, Jon Conrad, Carla P. Gomes, Bart SelmanUAI 2010 · OpenReview

Abstract

Managing renewable resources under radical uncertainty requires robust strategies that can withstand worst-case environmental fluctuations. This paper introduces a minimax framework that frames resource management as a zero-sum game against nature. By extending inventory control theory to biological systems, the authors prove that the optimal strategy is a pulsing $(S-s)$ policy. When applied to historical data from the Northern Pacific Halibut fishery, this approach demonstrates a 35% increase in guaranteed discounted revenue compared to traditional steady-state harvesting methods.

Introduction

The management of renewable resources, such as fisheries and forests, is complicated by density-dependent growth and unpredictable environmental shocks. Traditional management strategies typically rely on deterministic models that ignore uncertainty or risk-based models that assume known probability distributions for environmental events. However, managers often face "radical uncertainty," where the probabilities of specific outcomes are unknowable. In these cases, a policy must be robust against the worst-case scenario to prevent catastrophic depletion. This paper addresses the challenge of calculating mathematically sound, worst-case optimal policies for complex biological systems.

Methods: A Minimax Framework

The authors model resource management as a Markov Decision Process (MDP) structured as a two-person zero-sum game. In this setup, the manager aims to maximize utility (revenue minus costs), while "Nature" acts as an adversary, choosing environmental variables to minimize that utility.

A key technical contribution is the extension of $K$-concavity—a concept from inventory control literature—to the domain of biological growth. The authors prove that under certain conditions, the optimal solution is a non-stationary $(S-s)$ policy. This policy defines a lower threshold $s$; if the resource stock falls below this level, harvesting is suspended to allow for recovery. If the stock exceeds $s$, it is harvested down to a target level $S$. This structural insight significantly reduces computational complexity, allowing the MDP to be solved using a fast bisection-based approach rather than a brute-force search.

Key Insights: The Power of Pulsing

A central finding of this research is that "pulsing"—periodically closing the resource to allow for growth and then harvesting heavily—is more robust and profitable in a worst-case scenario than traditional steady-state management. This approach provides a guaranteed lower bound on utility, ensuring a minimum level of performance regardless of environmental conditions.

Optimal state and control trajectories illustrating the pulsing nature of the (S-s) policy over a 33-year horizon.

The "aha moment" of the study is the realization that a biological population behaves mathematically much like warehouse inventory with fixed reorder costs. This allows for the application of established inventory theorems to modern sustainability and conservation challenges.

Case Study and Results

The model was validated using 33 years of historical data (1975–2007) from the Northern Pacific Halibut fishery (Area 3A). The dataset included estimated biomass, annual harvest levels, and fishing effort.

Comparison of discounted revenue and loss for the optimal (S-s) policy versus historical and standard management strategies.

As shown in the results, the optimal $(S-s)$ policy achieved approximately 35% higher discounted revenue than historical harvest rates and the standard Constant Proportional Policy (CPP) used by many agencies. Furthermore, a "Rolling Horizon" strategy—a more practical, suboptimal version of the policy—also significantly outperformed historical management under worst-case realizations of stochasticity.

Conclusion

This research demonstrates that framing resource management as a game against nature can lead to more robust and efficient policies. By leveraging structural proofs, the authors provide a computationally efficient method for managing resources under radical uncertainty. While the current model assumes full observability of stock levels and focuses on single-species management, future work aims to incorporate Partially Observable MDPs (POMDPs) and extend the framework to multi-species ecosystems.

Summary updated

Abstract

In this paper we introduce a class of Markov decision processes that arise as a natural model for many renewable resource allocation problems. Upon extending results from the inventory control literature, we prove that they admit a closed form solution and we show how to exploit this structure to speed up its computation. We consider the application of the proposed framework to several problems arising in very different domains, and as part of the ongoing effort in the emerging field of Computational Sustainability we discuss in detail its application to the Northern Pacific Halibut marine fishery. Our approach is applied to a model based on real world data, obtaining a policy with a guaranteed lower bound on the utility function that is structurally very different from the one currently employed.
BibTeX
@misc{ermon2010playing,
  author = {Stefano Ermon and Jon Conrad and Carla P. Gomes and Bart Selman},
  title = {{Playing games against nature: optimal policies for renewable resource allocation}},
  year = {2010},
  eprint = {1203.3478},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1203.3478},
}
Figure