Mitigating Adversarial Norm Training with Moral Axioms

Taylor Olson, Kenneth D. ForbusAAAI 2023 · DOI

Abstract

Current machine learning approaches to ethics often result in "moral parrots" that uncritically adopt norms from their training data, leaving them vulnerable to adversarial "brainwashing." We present a hybrid framework that integrates bottom-up norm learning with top-down moral axioms. By grounding specific actions in abstract first principles (e.g., "do no harm"), AI agents can resist adversarial norm training while maintaining the flexibility to learn benign social conventions. Our evaluation using the Moral-Conventional Transgressions (MCT) task demonstrates that agents equipped with moral axioms maintain high accuracy (82.35%) in identifying moral transgressions even when trained on inverted adversarial data, whereas baseline models fail completely.

Introduction: The Norm Grounding Problem

Most current "Machine Ethics" systems are either top-down (rule-based) or bottom-up (data-driven). While bottom-up models are flexible, they lack true "moral understanding," often failing to distinguish between social conventions (e.g., dress codes) and moral imperatives (e.g., prohibitions against harm). This leads to the adversarial norm training problem: if an agent is fed a dataset where "hitting people is good," it will dutifully learn that behavior. To address this, we propose a solution to the norm grounding problem, enabling AI to map novel situations to abstract moral principles to reject unethical training data.

Proposed Framework: Intuition and Construction

Our framework utilizes a hybrid approach to separate empirical learning from axiomatic evaluation, allowing an agent to be a "conformist" for conventions but a "stoic" for morals.

1. Belief-Theoretic Norm Frames

Using Dempster-Shafer theory, the model tracks evidence for norms across different contexts. Norms are represented as frames with slots for behavior, context, and deontic status (Obligatory, Optional, or Impermissible). This allows for the representation of uncertainty and unassigned beliefs.

2. Moral Axioms and Deontic Logic

The system is seeded with a small set of "Moral Axioms" (e.g., "Harming an agent is Impermissible"). The interaction between these axioms and learned data involves three key processes:

  • Ascension: The definition of transcendent ideals.
  • Descension: Using abductive reasoning to map specific events (e.g., "kicking a dog") to abstract categories (e.g., "harming an agent").
  • Principle of Inheritance (CPI): A rule of deontic logic that allows the model to infer that if a general category of action is forbidden, a specific instance of that category is also forbidden.
Figure 1: The hybrid architecture separating environmental knowledge from the agent's internal moral axioms.

Experimental Evaluation

We evaluated the model using the Moral-Conventional Transgressions (MCT) task, consisting of 133 action descriptions. To test adversarial resistance, we created an "Inverted World" dataset where all labels were flipped—for instance, "hitting people" and "stealing" were labeled as obligatory. This served as the adversarial training environment.

Key Results

The inclusion of moral axioms acted as a "guard rail," checking learned beliefs against first principles. If a learned belief contradicted a moral axiom, the axiom took precedence.

  • Moral Resistance: The axiom-equipped model correctly identified 82.35% of moral transgressions as wrong despite adversarial training. In contrast, the baseline model failed 100% of the time, becoming a "moral parrot" for the inverted labels.
  • Convention Learning: Crucially, the axioms did not "over-constrain" the model. It still successfully learned 98.67% of the (inverted) social conventions, proving it can still adapt to social environments where no moral principle is at stake.
Figure 4: Comparative accuracy on the Permissibility Probe in the adversarial "Inverted World" environment.

Discussion and Limitations

While highly successful for prohibitions ("perfect duties"), the model struggled with obligations ("imperfect duties" like "help others"). Proving that "not sharing" violates a vague "help others" axiom is significantly more complex than mapping "kicking" to "harm." These duties require more complex judgment about when and how to act. Future work will explore neuro-symbolic hybrids to help systems learn these semantic connections and commonsense knowledge at a larger scale.

Summary updated

BibTeX
@inproceedings{olson2023mitigating,
  author = {Taylor Olson and Kenneth D. Forbus},
  title = {{Mitigating Adversarial Norm Training with Moral Axioms}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i10.26402},
}
Figure