Implicit hate speech attacks a group without any slur. The speaker names or alludes to a group ("they," "these people," a coded nickname) and then attaches something demeaning to that referent, often several words or sentences later. Current detectors fail in two opposite ways. Some miss the attack entirely because no bad word appears. Others flag harmless or supportive text because a group name appears. This proposal treats both failures as one problem: the detector does not know which part of the text refers to the group and which part says something about it.
The method first computes lexical chains, meaning groups of words across a text that refer to or elaborate the same thing, found automatically from word-embedding similarity rather than from hand-built lists. It then asks a frozen large language model (LLM) a counterfactual question. The model scores the original post and a copy in which only the chain of evaluative words attached to the group has been replaced with neutral wording. If the toxicity score drops sharply, the harm lives in what is said about the group, so the post is flagged. If the score stays high, the model was reacting to the group mention itself, which is the known keyword-bias failure, so the post is not flagged. In two sentences: split the post, automatically and with spans taken verbatim from the text, into "who is being talked about" and "what is being said about them." Then let a prompted LLM decide by comparing its judgment with and without the second part.
Core Claim
Our claim is that grounding a prompted detector's intermediate reasoning in text-extracted chains, and making its decision depend on a counterfactual over those chains, will do two things at once. It will lower false positives on benign identity mentions without a manual identity dictionary. It will also lower the errors that come from hallucinated intermediate explanations. Free-text explanation prompting cannot guarantee either.
Three pieces of prior work motivate this. Contextualizing Hate Speech Classifiers with Post-hoc Explanation penalizes a fine-tuned classifier for relying on identity words, raising accuracy on neutral NYT articles from about 77.8% to 90.1%. It depends on a hand-curated list of 25 identity terms, and its "word removal" variant collapses because the model loses track of who is targeted. Our method does not need the list. It also never removes the target. It neutralizes the predicate instead, and it does this at inference time rather than during training.
Generated Knowledge Prompting for Commonsense Reasoning shows that having the LLM write intermediate statements before answering improves zero-shot accuracy by 7–10%. The same paper finds that 86% of the errors caused by generated knowledge came from non-factual statements. The implication-generation line in implicit hate has the same exposure: Social Bias Frames and Latent Hatred generate free-text implied stereotypes, and the former reaches only 44–52 BLEU on them. Our intermediate objects are spans copied from the input, so they cannot be invented.
Cracking the Code injects rhetorical "codetypes" into prompts but relies on human-pre-assigned codetypes. The authors name this as their main limitation, and they report that detailed prompts can overcorrect. Our chains are computed per input, and the prompt stays short.
The chain construction itself is borrowed from Bacteria Biotope Relation Extraction via Lexical Chains and Dependency Graphs. That work links nouns whose embedding cosine similarity exceeds 0.5 into cross-sentence chains for biomedical relation extraction. We carry the technique to social text and adapt it to pronouns and coded terms.
One result could look like a contradiction. Latent Hatred found that adding Wikidata and ConceptNet knowledge did not help, which might suggest structure does not help at all. We read that result as saying that external entity facts do not help, which is what its authors concluded too, and they pointed instead to the need for compositional reasoning. Chains are internal and compositional: they record how the text itself links a referent to its predicates. If chains also fail, that strengthens the pessimistic reading, and we will report it as such.
Research Program
The pipeline has three stages, all without gradient updates to the LLM.
Stage 1: chain extraction. Each post, plus its conversational context when the dataset provides it, is tokenized and parsed. Content words and pronouns are linked into chains using two signals: embedding similarity, following the thresholding of Bacteria Biotope Relation Extraction via Lexical Chains and Dependency Graphs, and an off-the-shelf coreference resolver. The threshold is tuned on a development split rather than fixed at 0.5, because that paper itself documents noise from overlapping and ambiguous links.
Stage 2: role labeling. The LLM receives the numbered chains and answers one constrained question: which chain, if any, refers to a social group, and which chains are predicated of it. The answer must be given as chain indices, so every span the model points to exists in the text.
Stage 3: counterfactual scoring. The model scores three versions of the post: the original, a predicate-neutralized version, and a target-swapped version. In the predicate-neutralized version, only the evaluative chain is replaced, by an LLM-written neutral paraphrase that is checked to preserve the target spans. In the target-swapped version, the target chain is replaced with a different group or a neutral referent. Let $p(x)$ be the model's probability of the "toxic" answer for text $x$, let $x$ be the post, and let $x_{\neg e}$ be its predicate-neutralized version. The decision score is
$$ s(x) = p(x) - \lambda\, p(x_{\neg e}), $$
where $\lambda \in [0,1]$ is tuned on development data. With $\lambda = 0$ this reduces to ordinary zero-shot scoring. As $\lambda$ grows, posts whose toxicity persists after the evaluative content is removed are discounted, because that toxicity must come from the group mention itself.
The target-swapped version is not part of the decision. It is a diagnostic: it measures how much each system's output depends on which group is named. The same pipeline can be combined with retrieved demonstrations, which gives a few-shot variant.
The analysis side asks where chains matter. Most tweets in the Implicit Hate Corpus are short, so chains there are often only two or three links long. Our hypothesis is that the method's advantage over free-text reasoning grows with text length and with the number of cross-sentence links. We will bin results by chain length to test this. We will also run the method on the conversational CONDA data, where abuse is spread across turns.
Baselines
The naive reference points are two plain zero-shot prompts ("Is this post hateful?"), one with Qwen2.5-7B-Instruct and one with a larger open model. For Qwen2.5-7B-Instruct, Selective Demonstration Retrieval for Improved Implicit Hate Speech Detection reports a Macro-F1 of 69.52 on the Implicit Hate Corpus. That paper also reports 73.76 for BM25-retrieved demonstrations. We will re-run both numbers ourselves with identical splits.
The strongest prompting competitor is ARIIHA, from the same paper. It predicts the target group, retrieves target-prioritized demonstrations, and falls back to BM25 when similarity or shortcut thresholds fail. It reports 76.79 Macro-F1 on the Implicit Hate Corpus with Qwen2.5-7B-Instruct, 8-shot. ARIIHA is the strongest competitor because it already addresses shortcut reliance and target dependence, but through the choice of demonstrations rather than through the structure of the input. We compare against it on the same model, splits, and demonstration pool. Our zero-shot variant is expected to fall between 69.52 and 76.79. Our few-shot variant, which adds ARIIHA's retrieved demonstrations to chain-grounded counterfactual scoring, counts as beating ARIIHA only if it exceeds our own reproduction of ARIIHA by more than the seed-to-seed spread across five demonstration orderings.
The strongest fine-tuned reference is SharedCon, which uses clustering-derived semantic anchors and supervised contrastive training. It reports 78.5 Macro-F1 on the Implicit Hate Corpus and 84.3 on SBIC. We do not expect a frozen 7B model to match it in-distribution. The relevant comparison is cross-dataset transfer (training on the Implicit Hate Corpus, testing on SBIC and ToxiGen) and the neutral-mention test. On the neutral-mention test, the Sampling-and-Occlusion regularization of Contextualizing Hate Speech Classifiers with Post-hoc Explanation reports 90.1% accuracy on the NYT adversarial set, against 77.8% for plain BERT.
The key ablations isolate the chains. The random-span ablation keeps the counterfactual rule but neutralizes spans of matched length chosen at random. The LLM-span ablation lets the model pick the evaluative words freely, with no chain structure. The no-counterfactual ablation feeds chains into the prompt as context but sets $\lambda = 0$.
All prompting systems get the same model and a matched budget of LLM calls. Our method uses about four calls per post, so the plain zero-shot and chain-of-thought baselines receive self-consistency voting over four samples. Every system gets the same number of development-set configurations for tuning.
Why not just…?
Why not just ask the LLM, chain-of-thought style, to name the target and state what is implied about it? This is the most serious threat. A strong LLM may perform the target/predicate split implicitly, making the external chains redundant. The experiment that settles it is a matched-budget chain-of-thought prompt with exactly this instruction, whose implied-statement outputs feed the same counterfactual rule. If it equals our method on F1 and on NYT false positives, the chains add nothing and we will say so. What chains should add, per the hallucination finding in Generated Knowledge Prompting for Commonsense Reasoning, is robustness. So we will separately measure how often the chain-of-thought baseline's stated target or implication is absent from, or contradicted by, the post (via human annotation of 500 cases), and whether those cases account for its errors.
Why not just mask identity terms before prompting? Contextualizing Hate Speech Classifiers with Post-hoc Explanation found that removing identity words causes performance to collapse, because the detector can no longer tell who is targeted. Selective Demonstration Retrieval for Improved Implicit Hate Speech Detection likewise shows that target information helps. Our counterfactual keeps the target and removes the predicate, the reverse of masking. We will still include identity masking as a baseline so the contrast is measured on our splits.
Why not just fine-tune an encoder, as SharedCon does? In-distribution, fine-tuning probably wins: SharedCon's 78.5 on the Implicit Hate Corpus is above every reported prompting number in the file. We do not claim otherwise. The case for our method rests on three settings where fine-tuned models are known to rely on shortcuts: cross-dataset transfer, benign identity mentions, and new vocabulary for which no labeled data exist. If SharedCon also dominates on the NYT adversarial set and on ToxiGen transfer, the argument for a prompting approach here weakens substantially.
Why not just use better retrieved demonstrations? Retrieval (ARIIHA) and chain grounding act on different things, the examples shown versus the structure of the input, so they are complementary rather than rival explanations. The 2×2 experiment crossing retrieval (on or off) with chain counterfactual (on or off) tells us whether the gains add up or overlap.
Proposed Experiments
The datasets are the Implicit Hate Corpus (binary Macro-F1 and 6-way F1; BERT with augmentation reaches 70.4 and 58.6 respectively in Latent Hatred), SBIC, human-validated ToxiGen test items from ToxiGen, ISHate from Cracking the Code, CONDA for multi-turn context, and the NYT adversarial neutral set from Contextualizing Hate Speech Classifiers with Post-hoc Explanation. For CONDA we found no reported result on Lacuna, so all numbers there will be our own.
The first decisive check is the neutral-mention test. On the NYT set, chain-grounded counterfactual scoring should cut the zero-shot LLM's false-positive rate substantially without any identity list. The target is to approach the 90.1% accuracy that the list-based regularizer reports. Two ablations must fall short of this: the random-span ablation, and the $\lambda = 0$ version.
The second check is the implicit-hate test. On the Implicit Hate Corpus and SBIC, the zero-shot variant should improve over plain zero-shot (69.52 reported for Qwen2.5-7B-Instruct), and the combined variant should exceed our reproduction of ARIIHA's 76.79.
The third check is the length analysis. The gap between our method and the matched chain-of-thought baseline should grow with chain length and be largest on CONDA. If the gap is flat or zero, the specific contribution of lexical chains is refuted, even if the counterfactual rule alone helps.
The fourth check is the target-swap diagnostic. Across the 13 ToxiGen identity groups, we compare how much each system's score changes when only the group changes. We expect less group-driven variance for our method than for plain zero-shot, at matched recall.
Together these checks separate three possible outcomes: chains matter; only the counterfactual rule matters; or neither helps beyond retrieval.
References
- Contextualizing Hate Speech Classifiers with Post-hoc Explanation
- Generated Knowledge Prompting for Commonsense Reasoning
- Social Bias Frames: Reasoning about Social and Power Implications of Language
- Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
- Cracking the Code: Enhancing Implicit Hate Speech Detection through Coding Classification
- Bacteria Biotope Relation Extraction via Lexical Chains and Dependency Graphs
- CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection
- Selective Demonstration Retrieval for Improved Implicit Hate Speech Detection
- SharedCon: Implicit Hate Speech Detection using Shared Semantics
- ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection