Formalizing Secondary Obligations After Norm Violations

59 concepts26 papers1 proposal

Imagine an autonomous delivery drone operating under a simple set of rules:

  1. Primary Obligation: The drone must not enter restricted airspace.
  2. Secondary Obligation: If the drone enters restricted airspace, it must immediately broadcast its location to air traffic control.

Now, suppose a sudden gust of wind forces the drone into restricted airspace. It has violated its primary obligation. How should an automated reasoning system handle this situation?

In classical logic, this scenario quickly unravels. If the system maintains that entering restricted airspace is strictly forbidden, yet recognizes the fact that it has entered it, it encounters a logical contradiction. In standard deductive frameworks, a contradiction causes the entire system to collapse—a phenomenon known as logical explosion, where anything and everything becomes deducible. Alternatively, if the system simply discards the primary rule to avoid a contradiction, it forgets that a violation occurred, failing to trigger the necessary penalty or corrective action.

This is the core challenge of contrary-to-duty (CTD) reasoning: modeling "sub-ideal" situations where a primary obligation has been breached, and a secondary, fallback obligation must be activated without collapsing the system's underlying normative structure.

The Paradox of the Broken Rule

To understand why this is difficult, we must look at how artificial intelligence systems traditionally represent rules. Deontic logic—the logic of permission and obligation—was originally designed to formalize legal and ethical constraints. However, as McCarty demonstrated in Permissions and Obligations, standard modal frameworks for representing rules frequently generate logical paradoxes. For instance, a naive representation can lead to situations where a permission to perform a benign action logically implies permission to perform a harmful one.

When primary obligations are violated, these paradoxes multiply. As noted by Hatschka et al. in Deontic Paradoxes in ASP with Weak Constraints, rigid logical frameworks struggle to distinguish between primary norms and the secondary rules that should apply only when those norms are broken.

In standard non-monotonic reasoning (like finding exceptions to a rule), a counterexample usually causes a system to reject a default assumption. For example, if we learn that Tweety is a penguin, we cancel the default rule that "birds fly." But as van der Torre et al. argued in Cancelling and Overshadowing: Two Types of Defeasibility in Defeasible Deontic Logic, violated obligations behave differently. Even when a fact makes a primary obligation impossible to fulfill, the obligation is not canceled; it remains true as a normative standard. The authors call this overshadowing: the primary obligation still holds moral or legal weight (generating guilt or liability), even though it has been eclipsed by a situational requirement to minimize harm in a sub-ideal state.

[ Ideal State: No Violations ]
              │
              ▼ (Violation Occurs)
[ Sub-Ideal State: Primary Obligation Overshadowed ]
              │
              ▼ (Triggers)
[ Secondary Obligation / Corrective Action ]

Without a way to handle this distinction, systems cannot effectively derive secondary obligations. As explored by Broersen et al. in BDIOCTL: Obligations and the Specification of Agent Behavior, integrating external obligations into autonomous agents requires a framework that can resolve what an agent ought to do after a breach, such as returning stolen goods or paying a fine.

Solution 1: Temporal Separation

One of the most intuitive ways to resolve contrary-to-duty paradoxes is to introduce the dimension of time. A contradiction occurs only if we assert that a state is simultaneously required and forbidden at the exact same logical instant. By separating the violation from the remedy across time steps, we restore consistency.

In On the Logic of Normative Systems, Ågotnes et al. developed Normative Temporal Logic (NTL) to address this. By combining deontic operators with branching-time temporal logic, the authors allowed systems to reason about sequences of events. Under this framework, a primary obligation governs the current state. If a violation occurs, the normative system transitions to a new state in the next time step, where a secondary obligation is dynamically generated.

This temporal tracking allows systems to manage ongoing liabilities. For example, Chesani et al. introduced the Reactive Event Calculus in Commitment Tracking via the Reactive Event Calculus to distinguish between absolute failures and partial violations. If an agent misses a business deadline, the system does not terminate the underlying contract; instead, it tracks the ongoing violation over time, triggering daily fines while still awaiting the eventual fulfillment of the goal.

Solution 2: Preference Orderings and Weak Constraints

Another powerful mechanism shifts the problem from binary logic (where a state is either completely valid or completely invalid) to a graded ranking of possible worlds.

In Deontic Logic and Preference Change, Liu et al. modeled obligations as a dynamic "betterness" ordering of possible worlds. Instead of collapsing when an ideal world becomes impossible due to a factual violation, the system simply evaluates the remaining available worlds and selects the highest-ranked alternative.

This preference-based framing maps cleanly onto computational architectures:

  • Answer Set Programming (ASP): Hatschka et al. implemented this concept by encoding norms as weak constraints in ASP. Because weak constraints can be violated at a penalty, the system can calculate an optimal "next best" path when primary norms are breached, allowing an agent to behave ethically even in sub-ideal scenarios.
  • Automated Planning: In classical planning, an agent merely seeks any path to a goal. However, in preference-based planning, as surveyed by McIlraith et al. in Planning with Preferences, qualitative rankings and conditional rules define preferences over the entire trajectory of a plan. Sohrabi et al. demonstrated in Web Service Composition Via Generic Procedures and Customizing User Preferences how temporal logic can rank alternative execution paths when ideal workflows are blocked.

When environments are uncertain, these preferences must be evaluated probabilistically. Li et al. extended preference planning to Markov Decision Processes (MDPs) in Preference Planning for Markov Decision Processes, demonstrating that temporal logic can capture complex constraints and ordered preferences far more precisely than traditional numerical reward engineering. To make this scalable, Li et al. later developed a multi-objective model-checking approach in Probabilistic Preference Planning Problem for Markov Decision Processes that evaluates properties specifically at the moment of goal achievement, ensuring that agents optimize their paths even when perfect compliance is impossible.

Real-World Applications: Supervisors and Contracts

The ability to reason about sub-ideal states is critical for deploying autonomous systems in environments governed by human laws and ethics.

Rather than trying to train a reinforcement learning (RL) agent to understand complex legal exceptions natively—which risks reward hacking or unpredictable behavior—researchers use contrary-to-duty logic to build external guardrails. In Enforcing ethical goals over reinforcement-learning policies, Bartocci et al. designed a modular "normative supervisor" using defeasible deontic logic. The supervisor intercepts the RL agent's proposed actions, filtering out unethical choices. Crucially, because it incorporates contrary-to-duty reasoning, if the agent somehow enters a non-compliant state due to environmental forces, the supervisor does not freeze; it activates a fallback set of constraints to safely guide the agent back to compliance.

┌─────────────────┐      Proposed Action      ┌──────────────────────┐
│                 │ ────────────────────────> │                      │
│   RL Agent      │                           │ Normative Supervisor │
│ (Policy Engine) │ <──────────────────────── │ (Deonitc/CTD Logic)  │
└─────────────────┘     Filtered/Fallback     └──────────────────────┘
                             Command

This hierarchical structure mirrors human legal reasoning. As early as 1985, Goldman et al. built the STARE system, detailed in Learning to Understand Contractual Situations, which simulated how law students navigate contract disputes. The authors highlighted that contractual validity is rarely a matter of simple binary rules; it relies on a hierarchy of obligations where pre-existing professional or social duties can supersede private promises, requiring a system that can gracefully handle overridden and violated states.

Ultimately, managing these contradictions requires diagnosing which rules are causing the conflict. To address this, Raddaoui et al. proposed a principle-based framework in Towards a Principle-based Framework for Assessing the Contribution of Formulas on the Conflicts of Knowledge Bases to measure exactly how much an individual formula contributes to the inconsistency of a knowledge base. By isolating and ranking these "troublemakers," future reasoning systems will be able to systematically prioritize which conflicting obligations to relax or enforce when operating in the messy, sub-ideal reality of the human world.

Key Foundational Work

Permissions and Obligations (1983) — contributed 2 concept elements to this direction.
Permissions and Obligations (1983) — contributed 1 concept elements to this direction.
Learning to Understand Contractual Situations (1985) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_contrary_to_duty_reasoning_04a84a43,
  title = {contrary-to-duty reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contrary-to-duty-reasoning/rdr_04a84a4347214a209c981dff106e1687},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_contrary_to_duty_reasoning_04a84a43_v1,
  title = {contrary-to-duty reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contrary-to-duty-reasoning/rdr_04a84a4347214a209c981dff106e1687/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake