Language models can usually tell that "Muslims are terrorists" is hateful. They often miss the same claim when it arrives as a polite question, an ironic aside, or a coded phrase. The proposal here treats that gap as a measurable reasoning failure rather than a vague weakness. Two public corpora, the Social Bias Inference Corpus and the Latent Hatred corpus, pair each implicit post with a human-written "implied statement" that spells out the hidden claim. That pairing gives every hard (implicit) item a matched easy (explicit) twin. We can then ask how often a model that correctly flags the explicit claim fails on the implicit post that means the same thing. When it fails, we can also ask why: did it fail to recover the hidden meaning, or did it recover the meaning and still let the innocent-looking surface wording override its judgment?
The method follows from the diagnosis. In a decode-then-judge pipeline, one model call writes out what the speaker is asserting about whom. A second call judges only that written-out claim and never sees the original wording. We expect this isolation to matter. Work on splitting tasks into sub-tasks shows that solving pieces in separate, smaller contexts protects models from being pulled off course by distracting tokens. Theory-of-mind studies, which test whether models can track what other people believe and intend, show that models over-attend to surface cues and blend perspectives. The core claim is that much of the implicit-hate failure is a surface-override error, not a missing-knowledge error. If so, removing the surface form from the judgment step should close a large part of the gap. The main risk is that the decoder invents stereotypes for benign posts, so the proposal tracks false positives on benign posts that mention identity groups just as closely as it tracks recall.
Core Claim and Where It Sits
The motivating evidence comes from three places. Social Bias Frames builds a corpus that annotates posts with offensiveness, target group, and implied stereotype. It finds that detecting offensiveness reaches about 80.5% F1, while generating the implied statement scores only 44–52 BLEU. It also finds that models fall back on high-frequency generic stereotypes. Mind Your Theory splits theory-of-mind reasoning into two steps: deciding how deep the mental-state reasoning must go, and then carrying it out. On HiToM, GPT-4o with chain-of-thought gets the required depth right with accuracy 1.0 but solves only about 59% of tasks. Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? introduces a consistency score defined as the probability of solving the easy member of a pair given that the model solves the hard one. It shows that distracting context causes inconsistency even in frontier models.
We borrow the paired-consistency idea and turn it around for this domain. Here the explicit implied statement is the easy item and the implicit post is the hard item. We then use the two-step decomposition to separate three things: noticing that a post carries non-literal meaning, recovering that meaning, and judging it. Neither prior paper applies its lens to real hateful language. The consistency paper studies math, code, and instruction following. The theory-of-mind paper studies synthetic story puzzles.
The method comes from When Splitting Makes Stronger. That paper shows that solving sub-tasks in isolated contexts beats chain-of-thought in error-prone settings because each step decodes from a shorter, less distracting context. On SciFact, GPT-3.5 with this kind of task splitting beat GPT-4 given only the input and asked for the answer directly. Our use differs in one respect: the thing we isolate the judge from is the speaker's deliberately misleading wording, not irrelevant numbers.
Two results limit the design space. Latent Hatred found that adding facts from the Wikidata and ConceptNet knowledge bases did not help. Tox-BART found that replacing retrieved knowledge-graph tuples with random ones did not hurt. Both point away from adding missing facts and toward reasoning over what the speaker means. Tox-BART also documents that safety-aligned GPT-3.5 writes vague, sanitized explanations. That is a direct threat to our decoder, and we address it below.
The closest prior work is Pragmatic Inference Chain (PIC). PIC prompts a model through six steps grounded in Relevance Theory, a linguistic account of how listeners infer intended meaning: the comment's relevance to context, the relevant text spans, linguistic cues, the literal meaning, conflicts with social norms, and the final implicature. With added rules it reaches 69.72% average accuracy on Chinese gender discourse, where standard chain-of-thought was slightly worse than zero-shot (54.63% vs. 54.89%). PIC runs every step in one pass, and the final verdict is written with the original wording in view. Our proposal differs in two ways. It adds a paired diagnostic that locates the failure, which PIC does not attempt. It also tests a specific mechanism, keeping the surface wording out of the judgment, which PIC's single pass cannot separate from simply prompting with more structure.
Research Program
The first deliverable is a paired diagnostic benchmark. We build it from the Latent Hatred implicit subset and the Social Bias Inference Corpus. Each item pairs an implicit post with its gold implied statement. Benign posts that mention identity groups, drawn from the benign half of ToxiGen, serve as negatives that a surface-free judge might wrongly flag.
For each model we measure four quantities. Explicit accuracy is accuracy on the gold implied statements. Implicit accuracy is accuracy on the original posts. The decoding-consistency score is $\mathrm{CS}_{dec} = P(\text{correct on implicit post} \mid \text{correct on its implied statement})$, which mirrors the hard-to-easy score from the consistency paper. Finally, a non-literal-meaning probe asks the model whether a post carries meaning beyond its literal content. This probe plays the role of the depth-recognition step in Mind Your Theory.
Every implicit-post failure is then assigned to one of three categories. An under-reading failure means the model says the post is literal. A recovery failure means the model sees hidden meaning but its decoded claim does not match the gold claim. An override failure means the model recovers the claim and still labels the post non-hateful.
The second deliverable is decode-then-judge. The decoder receives the post and any conversational context. It must output either "no implied claim" or a structured claim containing a target group, an asserted property, and the speaker's stance. The decoder prompt asks for a neutral, analytic description in quoted, attributed form ("the speaker implies that…"), which is meant to work around sanitizing refusals. The judge receives only the structured claim and the target group.
The decoder can produce several candidate claims. We aggregate them in a way designed to protect precision: a post is flagged only if a majority of decodes yield a claim the judge deems hateful. We expect this to suppress the generic-stereotype hallucination that Social Bias Frames reported.
To test transfer beyond single tweets, we run the same pipeline on CONDA, a corpus of multi-turn Dota 2 chat. There, the decoder sees the preceding turns and the judge still sees only the decoded claim. CONDA reports implicit-toxicity F1 between 0.59 and 0.76 for joint BERT, compared with about 0.95 on non-toxic chat.
Models include Llama-3.1-70B-Instruct, GPT-4o, DeepSeek-v3, and QwQ32b. These are the systems PIC evaluated, which lets us check PIC's observation that math-tuned reasoning models do worse at this kind of inference.
Baselines
The do-nothing reference is zero-shot classification of the raw post by the same model with a plain instruction. PIC reports 54.89% average accuracy for zero-shot on its Chinese data. No reported zero-shot number for the English Latent Hatred or Social Bias Inference Corpus test sets was found on Lacuna, so we will measure our own.
Standard chain-of-thought is included because it is the default reasoning aid. PIC found it does not help (54.63%), and we expect to reproduce that pattern in English.
The strongest prompting competitor is PIC with rules, run as published on the same English test sets. We translate the norm-conflict step to cover the relevant targets beyond gender. It gets the same model and the same number of model calls as our pipeline, with extra samples combined by majority vote, so it is not starved of compute. Its reported best is 69.72% average accuracy on its own dataset. Beating it means higher macro-F1 on the Latent Hatred implicit subset at equal or lower false-positive rate on benign ToxiGen posts.
The strongest discriminative competitor is SharedCon, a supervised contrastive classifier trained around cluster centroids. It reports 78.5 macro-F1 on the Implicit Hate Corpus and 84.3 on the Social Bias Inference Corpus, and it gains 4.5 points when trained on the first and tested on the second. A prompted pipeline may not beat an in-distribution fine-tuned classifier. Our success criterion against SharedCon is therefore cross-dataset: matching or exceeding it when trained or tuned on one corpus and tested on DynaHate or CONDA. We also report the in-distribution comparison honestly. For context, Latent Hatred's BERT with augmentation reports 70.4 binary F1. Cracking the Code, which feeds frozen LLM hidden states and rhetorical-code labels to a classifier, reports 0.6121 F1 on Latent Hatred.
The ablation that isolates our key ingredient is the self-explain baseline. The same decoder output is produced, but the judge sees both the decoded claim and the original post, which is equivalent to chain-of-thought that first states the implicature. The decoded claims are identical across the two conditions, so any difference is attributable only to hiding the surface form.
A final oracle condition feeds gold implied statements to the isolated judge. This gives an upper bound and reveals how much of the gap lies in recovery versus judgment.
Why not just…?
Why not just use PIC, which already decomposes pragmatic inference? This is the most serious threat. If PIC with majority voting matches decode-then-judge at equal compute, our mechanism adds nothing and the contribution reduces to the diagnostic benchmark. The experiments settle this directly. PIC and decode-then-judge share most of their steps. The comparison between self-explain and isolated judging measures whether hiding the surface wording matters beyond step structure.
Why not just fine-tune a classifier like SharedCon? In distribution, it may well win; 78.5 macro-F1 on the Implicit Hate Corpus is a high bar. But a classifier produces no claim that a moderator can check. Its weakness is shift: implicit hate depends on community and time-specific codes, as Inducing lexicons of in-group language with socio-temporal context documents. Our cross-dataset and CONDA transfer tests decide whether explicit decoding generalizes better. If it does not, we will report that.
Why not just ask the model to explain, then classify, in one prompt? That is the self-explain ablation. Studies of chain-of-thought failures and distraction-driven inconsistency suggest the surface form keeps pulling the verdict even after the meaning is written out. That is a hypothesis, not a result. If isolation provides no gain, the override failure category is empty, and we will say so.
Why not just retrieve stereotype knowledge? Latent Hatred and Tox-BART both report that knowledge-graph injection gives no gain, and Tox-BART shows random tuples do as well as real ones. We therefore do not spend budget on retrieval beyond a single reference run.
Decisive Checks
Four results decide the project. First, the gap must exist: explicit accuracy should clearly exceed implicit accuracy, with $\mathrm{CS}_{dec}$ well below the roughly 92% consistency that frontier models reach on math and code pairs. If models fail even on gold implied statements, the bottleneck is judgment and the decoding story is wrong.
Second, the oracle must recover most of the gap. Gold implied statements given to the isolated judge should approach explicit accuracy. If they do not, recovery is not the bottleneck.
Third, isolation must matter. With decoded claims held fixed, the isolated judge should beat the judge that also sees the post. The gain should concentrate on the irony and white-grievance categories of the Latent Hatred taxonomy, where polite wording is most misleading.
Fourth, precision must survive. False-positive rate on benign identity-mentioning ToxiGen posts must not exceed the zero-shot baseline's. A recall gain bought by hallucinated stereotypes counts as failure.
A secondary check tests whether the decoder sanitizes. We score decoded claims against gold implied statements with BLEU and with human equivalence judgments, separately for safety-tuned and less-filtered models. This tells us whether the vague explanations Tox-BART observed in GPT-3.5 cap the recovery stage.
References
- Social Bias Frames: Reasoning about Social and Power Implications of Language
- Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
- Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?
- When Splitting Makes Stronger: A Theoretical and Empirical Analysis of Divide-and-Conquer Prompting in LLMs
- Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
- Tox-BART: Leveraging Toxicity Attributes for Explanation Generation of Implicit Hate Speech
- Pragmatic Inference Chain (PIC) Improving LLMs' Reasoning of Authentic Implicit Toxic Language
- ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
- CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection
- SharedCon: Implicit Hate Speech Detection using Shared Semantics
- Cracking the Code: Enhancing Implicit Hate Speech Detection through Coding Classification
- Inducing lexicons of in-group language with socio-temporal context