Abstract
Adaptive Management (AM) often relies on Mixed Observability Markov Decision Processes (MOMDPs) to navigate complex ecological and public health decisions. However, the resulting policies are frequently "black boxes" with thousands of rules, hindering human implementation. This paper introduces the K-N-MOMDP framework, which reduces model complexity by limiting observable states ($K$) and policy rules ($N$). By applying state abstraction and policy pruning, the framework generates simple, human-readable policy graphs that retain over 90% of the original model's value.
Introduction
In conservation and public health, decision-makers must often "learn by doing." Whether managing the habitat of a threatened species or controlling disease outbreaks through biocontrol, these Adaptive Management (AM) problems involve hidden dynamics and partial observability. MOMDPs provide a robust mathematical foundation for these problems, but their solutions—policy graphs—can grow to thousands of nodes. For a park ranger or health official, such complexity is a barrier to trust and practical application. The K-N-MOMDP framework addresses this interpretability-performance trade-off by shrinking these massive models into manageable flowcharts.
The K-N-MOMDP Framework
The authors propose a two-stage approach to simplify MOMDPs without significant loss in expected reward:
- K-MOMDP (State Abstraction): This stage reduces the number of fully observable states ($X$) to a user-defined maximum, $K$. It employs a transitive state abstraction function to group states with similar optimal actions and values into "bins," effectively simplifying the environment the manager perceives.
- N-MOMDP (Policy Pruning): This stage focuses on the policy itself, represented by $\alpha$-vectors. Three new algorithms—$\hat{\alpha}$-min-fast, $\hat{\alpha}$-min-p, and $\hat{\alpha}$-pruning—are introduced to select the $N$ most critical rules. This ensures the simplified policy remains as close to the original optimal policy as possible.
The combined K-N-MOMDP algorithm first abstracts the state space, solves the simplified model using a standard solver like SARSOP, and then prunes the resulting policy to $N$ vectors.
Evaluation and Results
The framework was tested on two real-world sustainability case studies: Gouldian Finch conservation and Dengue biocontrol (SIT).
Quantitative Performance
The results demonstrate that massive complexity reduction is achievable with minimal impact on performance. In the Gouldian Finch problem, the number of policy rules was reduced from 11,644 to just 6—a 99.95% reduction—while maintaining 98% of the original value.
As shown in the table above, the $\hat{\alpha}$ variants consistently achieve high value ($V_{\Gamma_N}$) with a very small number of vectors ($N$) compared to the standard SARSOP solver, often in just a few seconds.
Qualitative Interpretability
Beyond numerical performance, the primary goal is human readability. The framework produces policy graphs where each node represents a clear action based on observations.
For the Gouldian Finch (Figure 1A), the complex original policy is distilled into a 6-node graph involving actions like "Fire and grazing" or "Nesting boxes" based on population observations. Similarly, the SIT biocontrol problem (Figure 1B) is reduced to a 4-node decision tree for mosquito release.
Conclusion
K-N-MOMDPs bridge the gap between high-performance mathematical modeling and practical, interpretable decision-making. By demonstrating that a handful of rules can often perform nearly as well as thousands, this work empowers human managers to implement AI-driven strategies with confidence. Future research will focus on scaling these techniques to higher-dimensional problems and conducting behavioral studies to quantify the improvement in human trust and usability.
Reference: Ferrer-Mestres, J., Dietterich, T. G., Buffet, O., & Chadès, I. (2021). K-N-MOMDPs: Towards Interpretable Solutions for Adaptive Management. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16), 14775-14784.
Summary updated