Operationalising Rawlsian Ethics for Fairness in Norm Learning Agents

Jessica Woodgate, Paul Marshall, Nirav AjmeriAAAI 2025 · DOI

TL;DR

This paper introduces RAWL-E, a novel method for training AI agents to learn ethical social norms by operationalizing John Rawls's maximin principle. By incorporating an "ethics module" that prioritizes the well-being of the least advantaged, RAWL-E agents learn norms that promote fairness, social welfare, and robustness in multi-agent systems, addressing the "is-ought" gap in AI ethics.

What Problem Does This Paper Address?

In multi-agent systems (MAS), social norms emerge to guide behavior and enable coordination. However, if agents are purely self-interested, these norms can inadvertently lead to the exploitation or subjugation of certain agents, creating unethical outcomes. Existing approaches to promoting ethical behavior in AI often rely on descriptive statements of what is, rather than prescriptive principles of what ought to be, risking the perpetuation of existing unethical norms (the "is-ought" problem). This paper tackles the challenge of designing AI agents that can learn and adopt ethical norms, ensuring that emergent norms benefit society as a whole, particularly its most vulnerable members, by grounding their decision-making in established normative ethical principles.

Methods

The core contribution is the RAWL-E (Rawlsian Ethics Learning Environment) method. RAWL-E agents are equipped with three key modules:

  • Ethics Module (EM): This module operationalizes Rawls's maximin principle. It assesses the impact of an agent's actions on the well-being of all agents and generates a "self-directed sanction" (a reward shaping signal). This sanction is positive if an action improves the minimum well-being across the society, neutral if it doesn't change it, and negative if it worsens it. This directly addresses the "is-ought" gap by prescribing behavior based on ethical principles rather than observed outcomes.
  • Norms Module (NM): This module tracks learned behaviors and emergent norms. It stores behaviors along with their "fitness" (a combination of usage frequency and received rewards). When a behavior is adopted by a significant majority (90% convergence), it's considered an emerged norm and stored in a shared norm base.
  • Interaction Module (IM): This module uses a Deep Q-Network (DQN) architecture for reinforcement learning. It integrates the self-directed sanctions from the EM with environmental rewards to shape the agent's learning policy, encouraging it to pursue its goals while adhering to ethical norms.

The novelty lies in the explicit operationalization of Rawlsian ethics within a norm-learning framework, moving beyond agents that merely incorporate ethical decision-making without learning norms, or those that optimize for aggregate welfare without considering the least advantaged.

Datasets

The paper does not use pre-existing datasets for training. Instead, it introduces and evaluates its method within two novel simulated environments:

  • Capabilities Harvest: Agents can move freely but have different visibility of berries (some only visible to "tall" agents, others to "short" agents). Agents can learn to throw berries to each other.
  • Allotment Harvest: Agents are assigned specific "allotments" within a community garden and can only harvest within their area. This scenario creates inherent resource inequality.

These environments are designed to simulate situations with unequal resource distribution and opportunities, allowing for the evaluation of fairness-promoting norms.

Key Insights and Ideas

  • Operationalizing Maximin is Key to Fairness: Directly translating Rawls's maximin principle into an agent's decision-making process (via the EM) demonstrably leads to fairer outcomes than simply maximizing overall social welfare.
  • Bridging the "Is-Ought" Gap: By using normative ethics (Rawlsian principles) as a prescriptive guide, RAWL-E agents avoid the trap of learning unethical norms simply because they are prevalent.
  • Emergent Norms Reflect Ethical Principles: The learned norms in RAWL-E societies actively promote behaviors that benefit the least advantaged, such as cooperative actions like throwing berries, rather than purely self-serving ones.
  • Ethics as a Shaping Signal: Integrating ethical considerations as a reward-shaping mechanism (self-directed sanctions) effectively guides RL agents towards ethical behavior without sacrificing goal achievement.

Key Results

The paper evaluates RAWL-E agents against baseline DQN agents in the simulated harvesting scenarios. The results consistently support the hypotheses:

  • Reduced Inequality: RAWL-E societies exhibited significantly lower Gini indices for both well-being and resource distribution (e.g., for well-being in allotment harvest, Gini dropped from 0.2 to 0.1 with a large effect size of 1.58).
  • Improved Minimum Experience: The least advantaged agents in RAWL-E societies had higher minimum well-being (e.g., 10.82 vs. 7.18 in allotment harvest for well-being, with a large effect size of 3.09).
  • Higher Social Welfare: RAWL-E societies showed improved social welfare, measured by cumulative well-being (e.g., 59.80 vs. 20.60 in allotment harvest for well-being, with a medium effect size of 0.64).
  • Increased Robustness: RAWL-E societies survived longer, indicating greater robustness (e.g., average episode length of 48.19 vs. 47.36 in allotment harvest).

These improvements were more pronounced in the allotment harvest scenario, where resource distribution was more inherently unequal.

Limitations and Future Work

  • Scenario Simplicity: The authors acknowledge that the simulated environments are abstract and simplified. While this allows for clear demonstration of the method, future work should explore applying RAWL-E to more complex, real-world scenarios.
  • Single Ethical Principle: The current implementation focuses solely on Rawls's maximin principle. Future directions include operationalizing a variety of ethical principles and investigating how they might conflict or complement each other.
  • Reward Shaping: While effective, reward shaping can sometimes lead to implicit comparisons between objectives. Future work could explore combining explicit ethical promotion with explicit prevention of unethical outcomes.
  • Reproducibility: The authors provide their codebase and a detailed version of the paper for reproducibility.

Summary updated

BibTeX
@inproceedings{woodgate2025operationalising,
  author = {Jessica Woodgate and Paul Marshall and Nirav Ajmeri},
  title = {{Operationalising Rawlsian Ethics for Fairness in Norm Learning Agents}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i25.34837},
}
Figure