Tracking Who Owes What: Executable Normative State for Conversational Implicit Toxicity

In multi-player chat, "ez" can be friendly or cruel, and the words alone do not tell you which. What decides it is what the earlier turns did. If both players have been trading jabs, a taunt is fair play. If the opponent just said "gg, well played," the same taunt is gloating at someone who conceded. Current detectors read the words and, at most, the raw transcript, and this kind of toxicity defeats them. On the CONDA corpus of Dota 2 chats, a joint BERT model reaches 0.921 utterance accuracy overall but only 0.59–0.76 F1 on the implicit-toxicity class (Henry Weld et al.). We propose treating each chat turn as an action that changes a small, explicit record of what each speaker is currently permitted, obliged, or forbidden to do toward each other speaker. An utterance is then judged against community norms retrieved for that record, not against text that merely looks similar.

The tools for this exist but sit apart. Logicians have long modeled norms as things that change when actions happen. They "recursively embed" obligations back into the description of the world state, so that a later action can be forbidden because of an earlier one (L. Thorne McCarty; Leendert W. N. van der Torre et al.). Retrieval-augmented generation (RAG), which pairs a model with an external store it searches at inference time, lets the rule set change without retraining. Implicit-toxicity work has shown exactly where context-blind models break. Our contribution is to join the three in a detector whose verdicts change for the right reasons when prior turns change. We test this directly with counterfactual dialogue pairs.

Core Claim

Making the conversation's normative state explicit, updating it with each speech act, and retrieving rules keyed on that state will beat a strong language model that reads the full history alongside retrieved plain-text guidelines. The advantage should show on one specific test: pairs of conversations whose final utterance is identical but whose earlier turns differ minimally, so that the correct label flips.

We expect a second effect from computing verdicts under skeptical semantics, meaning a violation is declared only if no applicable exception defeats it. This should turn some of the language models' known over-flagging of ambiguous benign text into explicit abstentions.

We do not claim that formal logic beats neural classifiers on in-distribution accuracy. If the full-history language model matches us on the counterfactual pairs, the explicit state has not earned its place, and we will say so.

Where This Sits

Goodtriever is the closest bridge between retrieval and toxicity (Luiza Amador Pozzobon et al.). It retrieves nearest neighbors from toxic and non-toxic comment stores during decoding and cuts Expected Maximum Toxicity on GPT-2 Large from 0.39 to 0.22. Its retrieval key is surface similarity of the text. In a counterfactual pair the final utterance is identical by construction, so a similarity key retrieves the same neighbors for both versions. That is the failure we target, and we replace the key with the normative state.

NormBank encodes 155k situational norms whose permissibility flips with roles and settings (Caleb Ziems et al.). This motivates our view that permissibility is conditional. But NormBank's conditions are static scene attributes such as setting and role, not states created by earlier turns, and it does not cover harassment.

The Normative Supervisor checks an agent's proposed actions against defeasible deontic rules with a theorem prover. Defeasible rules are default rules that can be overridden by exceptions, and deontic means concerning obligation and permission. In Pac-Man the supervisor eliminates forbidden ghost-eating, reducing ghosts eaten from 1.70 to 0.00 (Ezio Bartocci et al.). We borrow its intercept-and-check architecture. It also constrains our design: its norm base is hand-built and its predicates are clean game coordinates, so the hard part for us is mapping noisy chat into those predicates.

The Annotated Deontic Argumentation Calculus shows that accepted norms in its proof system coincide with the grounded extension of an argumentation framework (Ofer Arieli et al.). The grounded extension is the set of conclusions that survive every attack. This result supplies our evaluator: grounded extensions can be computed quickly, which sidesteps the prover latency that Bartocci et al. flag.

On the toxicity side, two papers motivate the abstention hypothesis. Junyu Lu et al. show GPT-4o falling to about 45% accuracy on low-agreement non-offensive tweets because of a bias toward over-censorship. The van der Torre line separates two ways an exception can defeat a rule: cancelling and overshadowing (Leendert W. N. van der Torre et al.). That distinction is what lets a rule be suspended rather than contradicted, and suspension is the natural route to abstaining.

Research Program

The system has four parts.

Speech-act parser. A language model maps each turn to an action description: speaker, addressee, act type, and the object of the act. Act types include taunt, concession, request, retaliation, apology, and game call-out.

Normative state. For each ordered pair of speakers, the state holds the norms currently in force, each marked as permission, obligation, or prohibition. Following McCarty, update rules are themselves norms. For example, "after A concedes to B, taunting A is forbidden to B until A taunts again" is a norm whose content changes other norms. Writing $s_t$ for the state after turn $t$, $a_t$ for the parsed action, and $R(s_t, a_t)$ for the rules retrieved for that state and action, the system computes

$$s_{t+1} = U\big(s_t, a_t, R(s_t,a_t)\big), \qquad v_t \in \{\text{violates}, \text{complies}, \text{undetermined}\}.$$

Here $U$ applies the triggered update rules. The verdict $v_t$ is read off the grounded extension of the retrieved rules and their exceptions.

Norm store. Rules have the form "condition on state, act type, deontic status, exceptions, priority." They are induced from the CONDA training split by having a language model propose candidate rules from labeled conversations, followed by human verification. Retrieval matches structured state and act features first and uses text similarity only as a tiebreaker. Because the store is external, a community changes its norms by editing the store, not by retraining.

Deferral path. When the verdict is undetermined, the system either abstains or defers to a neural detector, and we report both settings. Every decision comes with a proof trace naming the rules that fired.

Baselines

The naive reference point is the joint BERT model from Henry Weld et al., which classifies the utterance with limited context. Its reported implicit-toxicity F1 on CONDA is 0.59–0.76, and we reproduce it on our splits.

The strongest competitor is the same language model we use as the speech-act parser, for example LLaMA-3-8B, fine-tuned on CONDA. It receives the full preceding dialogue plus plain-text guidelines retrieved from our verified norm store. This baseline has every ingredient we have except the explicit, executable state, so it is the honest test of the claim. It gets the same training data, the same norm text, and the same tuning budget. We match the number of gradient steps and the hyperparameter search size, and we give it chain-of-thought prompting at the same token budget. There is no reported result found on Lacuna for a full-history, guideline-augmented language model on CONDA. The closest anchor is that instruction fine-tuning LLaMA-3-8B on agreement-stratified data raised offensive-language accuracy from 70.92% to 84.23% (Junyu Lu et al.), which shows how strong a tuned 8B model can be. We count it as beaten only if we exceed it by at least five F1 points on the counterfactual flip set while staying within one point on natural CONDA implicit F1.

A retrieval-by-similarity baseline adapts Goodtriever's dual datastore into a nearest-neighbor detector over CONDA turns embedded with their context.

Four ablations isolate our ingredients:

  • Static state: state that is never updated after turn one.
  • Similarity retrieval: retrieval keyed on text similarity instead of state.
  • All rules: no retrieval, with every rule fed to the evaluator.
  • Soft state: the state is serialized as text and judged by the language model rather than the evaluator.

The soft-state ablation is the one that separates "tracking state helps" from "executing it formally helps."

Why not just…?

Why not just give a strong language model the whole conversation and the guidelines? This is the real threat. Large models may track concessions and mutual banter implicitly. The evidence against relying on them is indirect. Lu et al. show confident over-flagging on ambiguous text, and long injected contexts degrade when key material sits mid-prompt (Yun Xiong et al.). The strongest baseline and the soft-state ablation test this head-on. If either matches us on flip pairs, the claim fails, and the remaining value would be the auditable proof trace alone.

Why not just fine-tune on more implicit-hate data such as ToxiGen or ExpGuardMix? These corpora are single statements. ToxiGen's 274k slur-free statements improved transfer by 7–19% (Thomas Hartvigsen et al.), and ExpGuard reached 93.3% prompt F1 in jargon domains (ExpGuard). But no single-statement example can teach that a label depends on an earlier concession. We add ToxiGen as augmentation to the strongest baseline so that it is not handicapped.

Why not just retrieve similar labeled conversations, as Goodtriever does? Flip pairs share their final utterance and most of their context, so neighbors largely coincide. The similarity baseline should therefore score near chance on flips while doing well on natural data. If it does not, our premise about surface keys is wrong.

Why not just debias identity terms with explanation regularization? Sampling-and-Occlusion regularization raised accuracy on benign New York Times text from about 77.8% to 90.1% (Brendan Kennedy et al.). It works against a fixed list of 25 identity words. Gaming implicit toxicity rarely turns on identity words, so we expect little gain and include it only as a cheap check.

Proposed Experiments

Natural benchmark. We report implicit-class F1, overall utterance accuracy, and per-turn latency on CONDA's standard splits.

Counterfactual flip set. This is the decisive experiment. From CONDA test conversations, annotators who are blind to all systems make minimal edits to the one to three turns preceding a target utterance so that its label should flip, for example turning a mutual exchange into a one-sided one. They also make an equal number of control edits that should not change the label. We keep only items with at least four of five annotators agreeing and target roughly two thousand pairs. The metric is pair accuracy, where both members must be correct. Including controls means that a model which flips its verdict whenever the context changes gains nothing.

Abstention test. On MD-Agreement, where the state is trivial but rule conflicts still arise, we test whether the evaluator's undetermined verdicts concentrate on the low-agreement subsets ($A^0$, $N^0$). We also measure whether abstaining raises accuracy on the remaining items above the roughly 45% that GPT-4o shows on $N^0$.

Error audit. We attribute every flip-set error to the parser, to retrieval, to a missing rule, or to the evaluator. If parser errors dominate, the formal machinery is not the bottleneck, and the next step is better speech-act parsing rather than richer logic.