Abstract
Markov Decision Processes arise as a natural model for many renewable resources allocation problems. In many such problems, high stakes decisions with potentially catastrophic outcomes (such as the collapse of an entire ecosystem) need to be taken by carefully balancing social, economic, and ecologic goals. We introduce a broad class of such MDP models with a risk averse attitude of the decision maker, in order to obtain policies that are more balanced with respect to the welfare of future generations. We prove that they admit a closed form solution that can be efficiently computed. We show an application of the proposed framework to the Pacific Halibut marine fishery, obtaining new and more cautious policies. Our results strengthen findings of related policies from the literature by providing new evidence that a policy based on periodic closures of the fishery should be employed, in place of the one traditionally used that harvests a constant proportion of the stock every year.
Introduction
Managing renewable resources like fisheries or forests is a high-stakes balancing act. Aggressive harvesting (a risk-neutral approach) risks catastrophic population collapse, while overly timid management (a worst-case approach) sacrifices significant economic value. Traditional models often aim to maximize expected rewards, but in the context of sustainability science, the primary goal is ensuring the welfare of future generations. Standard risk-neutral models lack guarantees against rare but devastating outcomes, such as the total collapse of the Pacific Halibut fishery. This research seeks a "Goldilocks" middle ground: a mathematical framework that is cautious enough to prevent collapse while remaining economically viable.
Methodology
The authors model resource allocation as a discrete-time Markov Decision Process (MDP) characterized by non-linear dynamics and stochasticity. The framework introduces two key innovations:
- Exponential Utility Functions: Instead of maximizing raw revenue, the model maximizes an exponential utility function $U(x) = \exp(\gamma x)$, where $\gamma$ represents risk sensitivity. A small $\gamma$ approximates risk-neutral behavior, while a large $\gamma$ reflects a pessimistic, worst-case planning attitude.
- The $S-s$ Policy: The authors prove that for a broad class of these MDPs, the optimal solution is a nonstationary $S-s$ policy. If the resource stock falls below a threshold $s$, harvesting stops entirely to allow recovery. If it rises above $s$, the stock is harvested down to a target level $S$.
To establish this, the authors extend the concept of $K$-concavity from inventory control, providing new theoretical results on the behavior of $K$-concave functions under composition. This ensures that the proposed policy is a global optimum.
Case Study: Pacific Halibut Fishery
The framework was evaluated using 33 years of historical data (1975–2007) from the International Pacific Halibut Commission (IPHC) for Area 3A. This data was used to fit a stochastic version of the Beverton-Holt model, accounting for environmental shocks.
Key Results and Insights
The most striking insight is that "periodic closures"—completely shutting down a fishery for a year or more—is mathematically superior to the traditional Constant Proportional Policy (CPP), which harvests a fixed percentage every year.
The Risk-Sensitive (R-S) policies were compared against historical data and the standard CPP:
- Higher Revenue: In simulations, R-S policies achieved an average discounted revenue of approximately $113.5 million, significantly outperforming the $90.7 million generated by the traditional CPP.
- Enhanced Safety: Even in worst-case scenarios, the R-S policy outperformed the CPP ($90M vs $65M), demonstrating that risk-aversion leads to better long-term economic outcomes by preventing population crashes.
Furthermore, the exponential utility functions satisfy the "delta property," ensuring that the optimal policy depends only on the current state, making it practical for real-world implementation.
Discussion and Future Work
While the framework provides a robust method for single-species management, several avenues for future research remain. The current model focuses on a single resource, whereas real ecosystems are interconnected. Extending $K$-concavity to multidimensional spaces for multi-species or multi-region management presents a significant mathematical challenge. Additionally, while the model handles stochastic population growth, it assumes known ecosystem parameters. Future work could incorporate active learning to manage uncertainty regarding the underlying biological models.
Summary updated