A Tractable Leader-Follower MDP Model for Animal Disease Management

Régis Sabbadin, Anne-France VietAAAI 2013 · DOI

TL;DR

This paper tackles the challenge of coordinating individual farmers' disease control actions to manage animal epidemics. They introduce a novel, tractable "Leader-Follower" Markov Decision Process (LF-MDP) model that allows for efficient computation of optimal incentive strategies for a central organization, even with a large number of farms. This work offers a practical framework for designing adaptive, cost-effective disease management policies.

What Problem Does This Paper Address?

Managing endemic animal diseases, like Porcine Reproductive and Respiratory Syndrome (PRRS), is a complex challenge. While individual farmers are responsible for implementing control measures on their farms, the spread of disease is often a collective problem. A single farm's actions are rarely enough to contain an epidemic; the overall infection level in a region significantly impacts individual farm risk. To encourage coordinated action, organizations (like farmer cooperatives or government bodies) often use financial incentives.

However, modeling this scenario is difficult. Traditional approaches often frame it as a multi-agent decision problem, specifically a Leader-Follower Markov Decision Process (LF-MDP). The issue is that finding optimal policies in such games is computationally intractable when the number of "follower" agents (farmers) is large. This paper aims to bridge this gap by proposing a tractable LF-MDP model that can handle a large number of farms, making it practical for real-world disease management.

Methods

The authors adapt the standard LF-MDP framework by introducing two key simplifying assumptions:

  1. Identical Followers: All farmers are treated as indistinguishable from the organization's perspective, except for their infection status. This means they all follow the same policy given their state and the leader's incentive.
  2. Local Interactions: The disease spread and farmer decisions are primarily influenced by local factors and the global incentive, rather than complex inter-farm dependencies beyond the aggregate infection level.

These assumptions allow them to develop a polynomial-time algorithm for computing equilibrium policies. The core of their method involves a dynamic programming approach that interleaves:

  • Follower Nash Equilibrium Computation: For each state, they determine the optimal mixed strategies for farmers given the leader's incentive. This is framed as a bi-matrix game for each follower state.
  • Leader Optimal Strategy Computation: The leader then uses these follower strategies to optimize its own incentive decisions to minimize disease spread and associated costs.

They use a backwards induction algorithm, common in finite-horizon MDPs, but adapted for the leader-follower structure.

Datasets

Not discussed in the paper. The authors use a case study inspired by the PRRS disease, but no specific datasets are mentioned for training or evaluation.

Key Insights and Ideas

  • Tractability through Simplification: The paper demonstrates that by making reasonable assumptions about farmer homogeneity and local interactions, complex multi-agent problems can become computationally tractable.
  • Adaptive Incentives: The proposed model allows for "adaptive" incentive policies that change based on the current prevalence of the disease, a significant improvement over static incentive schemes.
  • Follower Indifference: The model highlights that followers' policies are often deterministic, meaning they will either always take a control action or never take it, depending on the incentive and their state.
  • Leader's Dilemma: The organization faces a trade-off: higher incentives might be needed to influence farmers, but these incentives also incur costs for the organization. The model helps find the optimal balance.
  • Bi-matrix Game Equivalence: For the specific S-I-S disease model, the follower coordination problem simplifies to solving bi-matrix games, which are more manageable than general n-player games.

Key Results

The authors evaluate their model on a simulated case study of PRRS control. Key findings include:

  • Deterministic Follower Policies: For the parameters used, follower policies were consistently deterministic.
  • Incentive Impact: Increased incentives generally lead to fewer infected herds.
  • Cost-Benefit of Incentives: While higher incentives can lead to better disease control, the leader's total cost can decrease in the long run due to reduced disease prevalence and associated costs, especially with a sufficiently long horizon.
  • Policy Stability: Equilibrium policies tend to be stationary (except for initial and final steps), making them interpretable and easy to implement.

Limitations and Future Work

  • Model Simplifications: The S-I-S model is a simplification of real-world disease dynamics. More complex models (S-E-I-R, etc.) would require computing k-player equilibrium policies, which is more challenging.
  • Homogeneous Followers: The assumption of identical followers might not hold in all real-world scenarios.
  • Partial Observability: The current model assumes full observability of the system state. Future work could extend this to handle partial observability (LF-POMDPs).
  • Non-homogeneous Rates: Extending the model to handle non-homogeneous disease transmission rates across farms is another promising direction.

Summary updated

BibTeX
@inproceedings{sabbadin2013tractable,
  author = {Régis Sabbadin and Anne-France Viet},
  title = {{A Tractable Leader-Follower MDP Model for Animal Disease Management}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2013},
  doi = {10.1609/aaai.v27i1.8476},
}
Figure