Moderation systems usually give one verdict per post: safe or violating. Many hostile posts are not uniform, though. A single comment can defend one group while attacking another, or praise an institution while harassing a named person. A post-level classifier then has to squeeze both attitudes into one score. The sentiment-analysis literature calls this failure sentiment leakage: the hostility aimed at one target bleeds into the judgment about another. We propose a small model that does something different. It reads a post and writes out a short list of records, each saying who is targeted, which exact words do the targeting, and whether that pairing violates policy. The "who" is produced by a pointer-generator. This is a decoder that, at each step, mixes two options through a learned probability: copying a word from the input, or generating a word from its own vocabulary. When the target is named in the text, including a new name or slang term the model has never seen, the decoder copies it. When the target is only implied, as with "they" or a coded reference to a group, it generates a canonical group label. The evidence span is always copied, so the highlight shown to a human reviewer is verbatim text by construction.
The central idea is that this copy-or-name choice is useful in its own right, beyond its effect on accuracy. A copied target is cheap to verify and generalizes to targets absent from training, which is the cold-start case. A generated target means the model is inferring who is meant, and that inference is exactly the kind of judgment human reviewers should see. We therefore use the model's own switch between copying and generating as the signal that decides which cases go to a human. The proposal tests three things: whether per-target decomposition reduces misattributed violations on multi-target posts, whether copying targets improves accuracy on target groups held out of training, and whether routing on the copy-or-name switch beats routing on classifier confidence at a fixed recall.
Core Claim
A moderation model that emits target-bound verdicts, with targets chosen by a copy-or-name mixture and evidence chosen by pure copying, will beat post-level classifiers and pure-generation extractors on three things. First, it will make fewer attribution errors on multi-target posts. Second, it will degrade less on target groups never seen in training. Third, it will expose a routing signal, the probability of naming rather than copying the target, that concentrates human review on the cases that actually need inference.
For the target slot, the decoder's distribution over the next target token $w$ at step $t$ is
$$P(w) = p_{\text{gen}}\,P_{\text{vocab}}(w) + (1-p_{\text{gen}})\sum_{i:\,w_i=w} a_i^t,$$
where $p_{\text{gen}} \in [0,1]$ is the learned gate, $P_{\text{vocab}}$ is the distribution over a small vocabulary of canonical group labels plus ordinary words, $w_i$ is the $i$-th input token, and $a_i^t$ is its attention weight. This is the formulation of Get To The Point, which introduced the gate for summarization. For the evidence slot we set $p_{\text{gen}}=0$, which forces copying. That is the same move Dr. Summarize made when it penalized generation in order to protect medical facts. The mean $p_{\text{gen}}$ over a record's target tokens is the routing score.
Why This, and Why Not the Obvious Idea
The idea one reaches for first at this intersection is a pointer-generator counterspeech writer: a model that copies verified statistics from retrieved documents into rebuttals of new hate narratives. The file lists this as a gap, since the counterspeech systems it describes lack copy gates. These include GitSearch, the RAG pipeline of LLM generated responses to mitigate the impact of hate speech, the RNN variational-autoencoder candidate generator of Generate, Prune, Select, and the FLAN-T5 prefix-tuned HiPPrO model of Counterspeech the ultimate shield!. We set that idea aside for two reasons. Modern language models already copy from their context well, so an added gate has little room to help. The file also notes a real tension: Generate, Prune, Select finds that counterspeech quality depends on diversity, which heavy copying works against. The case for the copy mechanism is much stronger for deciding what to flag and showing why. There, verbatim fidelity is the whole point, and the open-set nature of copying matches the cold-start problem: new targets and new slang appear before any labeled examples exist.
Each of the four directions plays a different role. The pointer-generator is the mechanism. Multi-target interference is the failure we aim to fix, and the aspect-sentiment literature is the evidence that it is real. Cold-start is the generalization regime we test. Moderation scalability is the deployment constraint that makes the routing signal matter.
Relation to Prior Work
The motivating evidence comes from aspect-level sentiment work. The Multi-grained Attention Network (MGAN) adds a loss that pushes apart the representations of aspects with different polarities. IACapsNet assigns features to per-aspect "capsules" (small vector groups) through iterative routing. RGAT-BERT routes sentiment along typed dependency edges such as subject and adjectival-modifier links. All three show that shared representations leak polarity across targets. All three are classifiers that assume the targets are given, and two of them depend on parsers or iterative routing, which the file notes break on informal text and are slow.
Role-Flipped Machine Reading Comprehension comes closest in output form. It extracts aspect–opinion pairs by alternately querying aspects for opinions and opinions for aspects. It is purely extractive, however, and cannot name a target that is only implied. Our copy-or-name target slot removes that limit, and we inherit its framing of pairs as the unit of output.
The mechanism comes from A Copy-Augmented Generative Model for Open-Domain Question Answering (FiD-PGN). It added a copy gate to a Fusion-in-Decoder reader, a question-answering model that encodes retrieved passages separately and fuses them in the decoder. The gate reused existing cross-attention and cost only 1,537 parameters, and it gained 9.6 exact-match points on questions whose answers never appear in training. That result is our reason to expect copying to help with unseen targets. We transfer the result from answer strings to moderation targets.
The deployment constraints come from moderation work. Supporting Human Raters found that LLM pre-filtering automated 41.5% of a review queue at 95.1% recall, and that highlighting keywords for raters raised their precision and recall by 9–11%. We ask whether highlights bound to a target do better than unbound keywords. ExpGuard shows that fine-tuned post-level guardrails far outperform commercial APIs on jargon-laden content. Its verdicts are still per post, though. Policy-as-Prompt shows that prompted LLMs change their edge-case decisions when only the prompt format changes. This constrains our strongest baseline: prompted extraction has to be evaluated under several prompt formats, not just the best one. GitSearch shows that cold-start delay is a real operational cost in crowd fact-checking. We address the classifier-side analogue of that delay.
Research Program
The model is a T5-size encoder-decoder. It reads a post and emits zero or more records in a fixed template: target, evidence span, and a verdict drawn from {violating, hostile-but-permitted, supportive, neutral}. The target slot uses the gated mixture above over a vocabulary extended with canonical labels for protected groups. The evidence slot copies only. Following FiD-PGN, the gate reuses decoder cross-attention, so the model adds almost no parameters. We also add the coverage penalty of Get To The Point so that the decoder does not emit the same record twice.
Data is the main cost of the project, and we are explicit about it: the file lists no benchmark with per-target moderation labels. We will build one by re-annotating posts drawn from HateDay, the Spotify Policy-as-Prompt set, and ExpGuardTest. Annotators mark every target, its evidence span, and a per-target verdict. We will oversample posts with two or more targets, in the spirit of MAMS, where every sentence has at least two aspects with conflicting polarity.
We add two diagnostic sets. The first is a paired leakage probe. We take a single-target violating post, add a clause that supports a second target, and check whether the verdict for the second target stays clean. These are artificial and are reported only as a mechanism check. The second is a held-out-target split built from the MultiCONAN target groups (Muslims, Women, LGBT+, Jews, Refugees): we train on four groups and test on the fifth, rotating through all five. For transfer to sentiment in general, we also train and evaluate the same architecture on MAMS, SemEval-2014 Laptop and Restaurant, and the Dong et al. Twitter set.
Baselines
The naive reference point is a fine-tuned post-level classifier. We use the released ExpGuard model and also fine-tune the same T5 backbone on post-level labels from our data. We turn their outputs into per-target predictions by assigning the post's verdict to every gold target. This is what current pipelines effectively do. ExpGuard reports 93.3% prompt F1 on ExpGuardTest, so this baseline is strong at the post level. Our model beats it only if per-target F1 and attribution error improve on the multi-target subset, while post-level F1 (computed from our records) does not drop by more than one point. Commercial APIs are included as a floor. They scored 0.3–0.6% F1 on ExpGuardTest in that paper.
The target-conditioned classifier is the standard way to do targeted sentiment. We run RGAT-BERT, and separately a BERT-size classifier with markers around the target, once for each candidate target. Candidates come from named-entity recognition plus a group lexicon. We also run a gold-target variant as an upper reference. On MAMS, RGAT-BERT reports 84.52% accuracy with gold targets. On the MAMS extraction task, RF-MRC reports 64.53% overall F1. Our model must come within two points of both numbers under the same splits to show the architecture is not weaker at the base task. On the moderation data, it must beat the NER-candidate version on end-to-end per-target F1, especially for implicit targets.
The strongest competitor is a frontier or open instruction-tuned LLM prompted to emit the same records as JSON, with a string-match filter that drops any evidence span not found verbatim in the post. We also fine-tune an extractor on the same T5 backbone that generates records with no copy gate. That second model is the key ablation, because it isolates the mechanism. Every model sees the same training data. Prompted models get the same number of development-set tuning trials as our model's hyperparameter search. Following Policy-as-Prompt, we report the mean and spread across plain-text, JSON, and YAML policy formats.
We also run ablations on our own model. One uses copy-only targets, which should fail on implicit targets. One uses generate-only targets, the same as the no-gate extractor. One replaces per-record verdicts with a single pooled verdict, which isolates the per-target decomposition. One routes on classifier confidence instead of $p_{\text{gen}}$, which isolates the routing claim.
Why not just…?
Why not just prompt a strong LLM to output target–span–verdict JSON and string-check the spans? This is the real threat. The string check does make evidence verbatim, and large models may attribute targets well. The proposal can still lose on accuracy and win elsewhere. There is no reported number on Lacuna for per-target moderation with prompted LLMs, so the head-to-head comparison is itself a finding. We expect three advantages to hold even if accuracy ties. The first is cost: LLM generated responses to mitigate the impact of hate speech reports $0.048 per GPT-4 response. The second is stability under format changes, where Policy-as-Prompt documents disagreement on edge cases despite similar average accuracy. The third is a calibrated routing signal, which the prompted model lacks. If the prompted LLM matches us on per-target F1, format stability, and routed recall, we will report that the mechanism is unnecessary at that scale and narrow the claim to cheap, small-model deployment.
Why not just run a target-conditioned sentiment classifier once per detected target? This is what the aspect-sentiment literature does, and it does address leakage. It has two limits. Its cost grows with the number of targets per post. It can only judge targets that a separate detector found, so implicit targets are missed entirely, and new slang targets are missed whenever the lexicon lacks them. The held-out-group split and the implicit-target subset of our annotation measure both gaps directly. If the per-target classifier with NER candidates matches us on those subsets, the decoder's advantage collapses to throughput alone.
Why not just tag spans with a token-level labeler and pair them afterward? A tagger is also extractive and open-set, so it shares the copy mechanism's cold-start benefit for named targets. It cannot produce a target that does not appear as a span, and it needs a separate pairing step to bind evidence to target. That pairing step is where attribution errors in multi-target posts would reappear. We include a tagger plus pair-classifier on the same backbone. The comparison on attribution error and implicit targets settles whether joint decoding is doing work.
Proposed Experiments
The first decisive check is attribution on multi-target posts. We measure the rate at which a violating verdict lands on the wrong target, on the annotated multi-target subset and on the paired leakage probes. The claim fails if the pooled-verdict ablation and the post-level classifier are within noise of the full model.
The second check is cold start. On each held-out MultiCONAN group, and on posts naming entities absent from training, we compare the full model against the no-gate extractor. We report the per-target F1 gap between seen and unseen targets for each. FiD-PGN's 9.6-point no-overlap gain in question answering is the analogue we expect. We have no prior number for moderation, and a gap smaller than two points would count against the mechanism.
The third check is routing. We fix recall of violating targets at 95%, and measure the fraction of the queue that can be decided automatically when routing on $p_{\text{gen}}$ versus classifier confidence versus the LLM's self-reported uncertainty. The 41.5%-automated-at-95.1%-recall figure from Supporting Human Raters was measured on a different, internal queue. We report it as context, not as a direct comparison.
The fourth check is a rater study. Raters see either target-bound highlights from our model or unbound keyword highlights like those in that paper, and we measure their per-target precision, recall, and time. Target-bound highlights must improve per-target accuracy without increasing review time.
Together, these four checks separate three possibilities: that per-target decomposition helps, that copying targets helps with cold start, or that a well-prompted LLM makes both unnecessary.
References
- Get To The Point: Summarization with Pointer-Generator Networks
- Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures
- LLM generated responses to mitigate the impact of hate speech
- Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech
- Counterspeech the ultimate shield! Multi-Conditioned Counterspeech Generation through Attributed Prefix Learning
- Multi-grained Attention Network for Aspect-Level Sentiment Classification
- Capsule Network with Interactive Attention for Aspect-Level Sentiment Classification
- Investigating Typed Syntactic Dependencies for Targeted Sentiment Classification Using Graph Attention Neural Network
- Self Question-answering: Aspect-based Sentiment Analysis by Role Flipped Machine Reading Comprehension
- A Copy-Augmented Generative Model for Open-Domain Question Answering
- Supporting Human Raters with the Detection of Harmful Content Using Large Language Models
- ExpGuard: LLM Content Moderation in Specialized Domains
- Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Models