Exploring Equity of Climate Policies Using Multi-Agent Multi-Objective Reinforcement Learning

Palok Biswas, Zuzanna Osika, Isidoro Tamassia, Adit Whorra, Jazmin Zatarain Salazar, Jan H. Kwakkel, Frans A. Oliehoek, Pradeep K. MurukannaiahIJCAI 2025 · DOI

TL;DR

This paper introduces JUSTICE, a novel framework that combines Integrated Assessment Models (IAMs) with Multi-Objective Multi-Agent Reinforcement Learning (MOMARL) to explore equitable climate policy options. By modeling diverse policy actors and considering multiple objectives simultaneously, JUSTICE offers a more realistic and nuanced approach to climate policy design than traditional IAMs.

What Problem Does This Paper Address?

Addressing climate change requires coordinated global policy efforts, but current tools, like Integrated Assessment Models (IAMs), often fall short. Traditional IAMs typically optimize for a single objective, leading to policy recommendations that can perpetuate inequalities and overlook crucial trade-offs between economic growth, temperature goals, and climate justice. This simplification has been criticized for disadvantaging developing nations and fueling disagreements in international negotiations. While some research has explored multi-objective or multi-agent approaches separately, no existing framework effectively combines both to capture the complex, real-world interactions and diverse preferences of policy actors.

Methods

The core innovation is the development of JUSTICE, a framework that integrates a climate-economy model with Multi-Objective Multi-Agent Reinforcement Learning (MOMARL). This MOMARL approach models the problem as a Multi-Objective Multi-Agent Markov Decision Process (MOMAMDP) with a team reward structure.

  • Multi-Agent: The model represents 12 distinct macro-regions as individual agents, each making decisions about emission control and savings rates. This allows for a more realistic simulation of interactions between different policy actors.
  • Multi-Objective: Agents optimize for multiple, often conflicting, objectives. In this case, the primary objectives are maximizing global economic output and minimizing global temperature rise.
  • Reinforcement Learning: The agents learn optimal policies through trial and error, interacting with the environment and receiving rewards based on their actions. The paper uses Multi-Objective Multi-Agent Proximal Policy Optimization (MOMAPPO) for training.
  • Integrated Assessment Model (IAM): JUSTICE builds upon the RICE50+ model, incorporating its economy, damage, and abatement modules, and integrates them with the FAIR climate model for more accurate climate sensitivity assessments.

Datasets

  • Shared Socioeconomic Pathway (SSP) dataset: Used for exogenous regional economic growth, carbon intensity, and population projections. Specifically, SSP-2 was used for initialization.
  • Representative Concentration Pathway (RCP) dataset: Used to inform emission trajectories for greenhouse gases and aerosols.

The paper does not introduce a new benchmark dataset but rather uses existing datasets within its novel MOMARL framework.

Key Insights and Ideas

  • Equity is sensitive to policy framing: The choice of objectives and the number of agents significantly impact the equity outcomes of climate policies, as demonstrated by the GINI index analysis.
  • MOMARL offers a richer policy landscape: Unlike single-objective IAMs that provide a single "optimal" policy, JUSTICE generates a Pareto front of equitable policy options, allowing policymakers to explore trade-offs.
  • Real-world complexity matters: Modeling diverse agents and multiple objectives is crucial for understanding the nuanced dynamics of climate negotiations and for designing policies that are more likely to be accepted and effective.
  • Early mitigation is key: JUSTICE's policies favor rapid near-term mitigation, aligning with criticisms of traditional IAMs that may undervalue early action.

Key Results

  • JUSTICE generates a Pareto front of 22 policies: This contrasts with the single solution provided by the RICE50+ model.
  • The JUSTICE "Compromise Policy" dominates RICE50+: While RICE50+ offers a balance, the compromise policy from JUSTICE provides a slightly better outcome across objectives.
  • Equity analysis shows variation: The "Climate Policy" and "Economic Policy" generated by JUSTICE resulted in higher emissions inequality compared to the "Compromise Policy." The RICE50+ policy also showed increasing inequality over time.
  • Fairness requires explicit equity objectives: While JUSTICE explores equity through multi-agent and multi-objective setups, the authors note that explicitly defining equity objectives would further enhance its fairness analysis.

Limitations and Future Work

  • Team Reward Simplification: The current MOMARL setup uses a team reward, assuming collaboration. Future work could explore individual reward structures to better reflect countries' differing interests.
  • Linear Utility Function: The model assumes a linear utility function for multi-objective decision-making. Relaxing this assumption to explore non-linear preferences is a key direction.
  • Scalability: Analyzing 12 macro-regions is an improvement, but more fine-grained resolution for equity insights would require significant speedups in MOMARL algorithms.
  • Communication: The current model assumes cooperation. Future work could incorporate more complex communication strategies to model real-world bargaining and trade.
  • Policy Realism: The model assumes mitigation begins immediately. Future research could explore more realistic policy implementation challenges and geopolitical dynamics.
  • Open-Source Framework: The authors emphasize the open-source nature of JUSTICE, encouraging further exploration and development by the research community.

Summary updated

BibTeX
@inproceedings{biswas2025exploring,
  author = {Palok Biswas and Zuzanna Osika and Isidoro Tamassia and Adit Whorra and Jazmin Zatarain Salazar and Jan H. Kwakkel and Frans A. Oliehoek and Pradeep K. Murukannaiah},
  title = {{Exploring Equity of Climate Policies Using Multi-Agent Multi-Objective Reinforcement Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1064},
}
Figure