Much hateful text contains no hateful words. "I respect every culture, but some of them just don't belong here" carries its attack in the turn from the first clause to the second, not in any single token. A natural way to catch this is to split a post into clauses and reason about how they combine. Discourse researchers already do this. They cut text into elementary discourse units (EDUs, roughly clauses) and then parse the relations between them, such as concession or contrast. They also know the weak point of that pipeline. When the first step places a boundary wrongly, every later step inherits the mistake and cannot undo it. This is called cascading error propagation. Social media text has no reliable punctuation or casing, so it is where this cascade should be worst. It is also where implicit toxicity lives.
This note proposes a toxicity classifier that never commits to one segmentation. It scores toxicity for every candidate clause and sums over all ways of cutting the post into clauses, each weighted by a learned boundary model. The exact sum is computed by dynamic programming. Because the boundary model is trained through the toxicity loss, it learns to place boundaries where they matter for harm, such as the "but" in a concession. The work has two parts. First, we measure whether segmentation errors at rhetorical pivots actually cause missed implicit hate, which nobody has quantified. Second, we test whether averaging over boundaries removes that failure while keeping what clause structure offers: clause-level localization of the attack, and less reliance on identity words.
Core Claim
Our hypothesis is that much of the gap between explicit and implicit toxicity detection comes from intra-sentence rhetorical pivots. A model that represents clauses gains on these cases only if it does not hard-commit to a segmentation. Hard-committed pipelines should lose most of that gain, and should lose it precisely on posts where the segmenter misses the pivot boundary.
Each of these claims can be tested and could fail. A flat transformer may already handle pivots. Clause structure may not help even with gold boundaries. Or an ordinary segmenter may be accurate enough on toxic text that marginalizing changes nothing. The experiments are designed so that each outcome is visible.
Where This Sits
Two discourse papers motivate the method by showing that clause structure helps affective classification. Better Document-level Sentiment Analysis from RST Discourse Parsing weights discourse units by tree depth and composes sentiment over rhetorical trees, raising lexicon accuracy on Pang & Lee from 68.3% to 72.6%. Aspect-based Sentiment Analysis through EDU-level Attentions attends over EDUs produced by an off-the-shelf segmenter. It reaches 70.28% versus BERT's 66.98% on Rest14-Hard, and the authors note that accuracy is capped by that segmenter. Both papers use a fixed pipeline on edited review text. We study unedited abusive text and remove the fixed pipeline.
Two other papers supply the idea that segmentation and structure should share computation. A Unified Linear-Time Framework for Sentence-Level Discourse Parsing shares one encoder between a segmenter and a parser. A Conditional Splitting Framework for Efficient Constituency Parsing treats segmentation as one kind of split decision inside a single top-down decoder. Both still output one best segmentation, both are trained on newswire (RST-DT), and both treat boundaries as answers to a discourse question rather than a toxicity question. We keep their shared-encoder idea. The changes are to keep a distribution over segmentations and to supervise it with harm labels.
On the toxicity side, three methods work at the token or utterance level. Contextualizing Hate Speech Classifiers with Post-hoc Explanation penalizes attribution to 25 identity words. Why Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech Detection pre-finetunes a model to reconstruct masked human rationale tokens. CONDA labels gaming chat jointly at the utterance and token level. None of these has a clause unit or a relation between clauses.
Latent Hatred both motivates and constrains the proposal. It reports that expletives appear in only 3.2% of posts. It also reports that adding Wikidata and ConceptNet "didn't significantly boost performance", and concludes that the problem is compositional. We take that conclusion at face value and test one specific compositional mechanism.
A Decision-Based Approach to Rhetorical Parsing and the DISRPT 2023 shared task supply the skeptic's prior. Boundary errors cap downstream accuracy, and the automatic-sentence-split track degrades downstream tasks. That is exactly why we do not trust a single segmentation.
Research Program
Annotation and error measurement. We annotate about 1,000 posts. Roughly half come from Latent Hatred implicit-hate and non-hate posts that mention protected groups, and the rest from CONDA implicit-toxicity utterances. Two annotators mark EDU boundaries and mark which clause, if any, carries the attack. They also mark whether a pivot relation (concession, contrast, or a rhetorical question) connects that clause to its neighbor.
On this set we run DisCut from DisCut and DiscReT: MELODI at DISRPT 2023. It is the strongest segmenter in the file, with a mean treebanked-track F1 of 91.87 across DISRPT corpora. This gives the first measurement of segmenter quality on toxic text, which section 4 of the file identifies as unstudied. We then test whether a hard-boundary toxicity pipeline is more likely to miss hate when the segmenter merged the pivot boundary than when it found it, holding post length and keyword presence fixed. If that conditional error gap is near zero, the cascade story is wrong for this domain. We would report that result and drop the structural method.
The model. A shared transformer encoder produces two things for every span $u$ of a post $x$. The first is a clause toxicity probability $f(u)$, computed from the span's contextual representation, so a clause's score already reflects its neighbors. The second is a boundary score, initialized by distilling DisCut's boundary probabilities. These scores define a span-factored distribution $p(s \mid x)$ over segmentations $s$. We combine clause scores with a noisy-OR rule: the post is non-toxic only if every clause in the segmentation is non-toxic. The post-level probability is then
$$ p(\text{non-toxic} \mid x) = \sum_{s} p(s \mid x) \prod_{u \in s} \bigl(1 - f(u)\bigr). $$
This sum factorizes over adjacent spans, so a semi-Markov forward recursion computes it exactly in time linear in length times maximum span width. To model pivots explicitly, we add a pairwise term on adjacent spans that scores a contrast or concession transition. The recursion stays first-order and therefore exact.
Training combines two losses: cross-entropy on post labels, and a weight-decayed agreement loss toward the distilled boundary prior so that boundaries stay linguistically sensible. Where the annotated subset provides clause labels, we add direct supervision. At test time the model outputs two things: the post probability, and the posterior probability that each clause carries the attack. The second output is the moderator-facing explanation.
Conversations. For CONDA, previous turns are concatenated with fixed turn boundaries, and segmentation is marginalized only inside turns. This matters because CONDA's own example, "not a good pudg", is toxic only given an earlier "worst hookshot ever".
Baselines
The naive reference point is a flat RoBERTa-large classifier fine-tuned on each dataset with no notion of clauses. The closest reported figures come from Latent Hatred: BERT with augmentation reaches binary F1 70.4 and 6-way F1 58.6. CONDA reports Joint BERT at utterance classification accuracy 0.921, with implicit-toxicity F1 between 0.59 and 0.76. We rerun this baseline ourselves on identical splits rather than relying on the published numbers alone.
The hard-pipeline baseline segments with DisCut and then applies EDU-level sparse attention in the style of Aspect-based Sentiment Analysis through EDU-level Attentions. This baseline embodies the cascade, and the hypothesis predicts it fails specifically on posts with a missed pivot boundary.
The strongest structural competitor is a multi-task flat model. Its shared encoder is trained with both a toxicity head and a boundary-tagging head, following the shared-encoder design of A Unified Linear-Time Framework for Sentence-Level Discourse Parsing. A cheaper version inserts DisCut's predicted boundaries into the input as marker tokens. These two baselines separate "the encoder knows about clauses" from "the model computes over clause hypotheses".
For identity-word false positives, the strongest competitor is SOC regularization from Contextualizing Hate Speech Classifiers with Post-hoc Explanation. It reports 90.1% accuracy on the NYT adversarial set against roughly 77.8% for plain BERT. On HateXplain, the competitor is Masked Rationale Prediction. We found no reported result on Lacuna for its numeric HateXplain scores, so we reproduce it.
Every system uses the same encoder size and the same training data, and receives the same hyperparameter search budget of 20 trials per method selected on development F1. Each is run over five seeds.
Three ablations isolate the key ingredient. The first replaces the sum over segmentations with the single best (Viterbi) segmentation. The second freezes the boundary model at its distilled prior. The third supplies gold boundaries, as an upper bound on what clause structure can offer.
Beating the strongest competitor means two things. The model must exceed the multi-task flat model on implicit-class F1 for both Latent Hatred and CONDA, with seed-level confidence intervals that do not overlap. It must also match SOC on NYT accuracy without SOC's identity-word list. A smaller result is still informative: if the model ties on F1 but gives clearly better clause localization and a flatter degradation curve, that is a narrower claim, and we would report it as such.
Why not just…?
Why not just fine-tune a larger transformer or LLM? This is the most serious threat. The CONDA and Latent Hatred numbers show flat models struggling on implicit cases, but those are BERT-scale results, and a larger model might absorb pivots on its own. We therefore include a flat model with matched parameter count. The test that settles the question is the pivot-stratified evaluation. If the flat model's errors do not concentrate on concession and contrast posts, the structural hypothesis loses its main motivation, and we say so.
Why not just split on sentences or punctuation? The DISRPT 2023 findings show that automatic sentence splitting degrades downstream discourse tasks. The sentiment papers above show that the decisive pivots sit inside sentences. Chat text such as CONDA often has no punctuation at all. A sentence-split variant of the hard pipeline is cheap, so we run it, and expect it to merge exactly the pivot clauses.
Why not just add boundary tokens or a segmentation auxiliary head? This is the alternative most likely to match the proposal on post-level F1. It is the reason our central ablation compares the full model against its single-best-segmentation version and against the multi-task model. If those ablations equal the full model, then summing over segmentations is unnecessary, and the contribution reduces to the error measurement plus the cheaper method.
Why not just regularize away identity words as SOC does? SOC addresses false positives on benign mentions of protected groups, but by construction it does nothing for posts with no identity word or slur. Latent Hatred's 3.2% expletive rate suggests such posts are common. We also test whether SOC combined with our model stacks, since the two operate at different levels.
Decisive Checks
Four results decide the proposal.
The first is the conditional error gap on the annotated set: how much more often hard pipelines miss hate when the pivot boundary is lost.
The second is a controlled perturbation curve. We progressively delete or shift gold boundaries near pivots and plot implicit F1 against boundary corruption for the hard pipeline, the single-best ablation, and the full model. The hypothesis predicts a steep slope for the first two and a shallow one for the full model.
The third is clause-localization accuracy against the annotators' attack-clause labels, compared with token attributions from SOC and Masked Rationale Prediction projected to clauses.
The fourth is the gold-boundary oracle. If oracle clauses do not beat the flat model, clause structure does not help this task, and the marginalization machinery is moot.
References
- Better Document-level Sentiment Analysis from RST Discourse Parsing
- Aspect-Based Sentiment Analysis Through EDU-Level Attentions
- A Unified Linear-Time Framework for Sentence-Level Discourse Parsing
- A Conditional Splitting Framework for Efficient Constituency Parsing
- Contextualizing Hate Speech Classifiers with Post-hoc Explanation
- Why Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech Detection
- CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection
- Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
- A Decision-Based Approach to Rhetorical Parsing
- The DISRPT 2023 Shared Task on Elementary Discourse Unit Segmentation, Connective Detection, and Relation Classification
- DisCut and DiscReT: MELODI at DISRPT 2023