Spoil First, Type Second: Choosing a Clickbait Spoiler's Shape by Whether the Article Supports It

Clickbait headlines hide a fact to make readers click. "Clickbait spoiling" systems undo this by posting the hidden answer directly. Every strong system so far works in two steps. First, a classifier guesses from the headline what shape the answer will take: a single phrase, a passage of a few sentences, or a multi-part list. Second, a specialized model extracts or writes an answer of that shape. This ordering causes two failures that are really one failure. When the first step guesses wrong, the second step is forced to produce an answer of the wrong shape. If the second step is a generative language model, it does what such models do when the evidence is missing: it fills the shape with plausible fabrications. A model told to produce "5 foods" will list five foods, whether or not the article names five.

We propose to reverse the order. The system drafts a candidate spoiler in each of the three shapes. It checks every unit of every candidate (each phrase, each sentence, each list item) against the article for support. It then picks the shape whose candidate the article actually backs up. The spoiler type stops being a guess made from sensationalist headline wording and becomes a conclusion drawn from evidence. The same grounding check that picks the type also screens out invented content. The central claim is that, in this task, choosing the type and preventing hallucination are the same problem. Treating them jointly should help most on multi-part spoilers, which are the weakest class for type prediction and the class where generators most often invent items.

Why this is the right junction

The motivating evidence comes from both directions. On the spoiling side, Clickbait Spoiling via Question Answering and Passage Retrieval defined the three types and showed that type errors cascade into extraction errors. The end-to-end pipeline spoiled only 46–50% of test cases correctly. Clark Kent at SemEval-2023 Task 5 and the SemEval-2023 Task 5 overview report that the multi-part class is ambiguous even for human annotators who have not yet seen the spoiler text. That observation is the hinge of this proposal: if humans need the answer to know its shape, a classifier that predicts the shape before any answer exists is working with the wrong information. On the hallucination side, Saved You A Click found that abstractive spoilers produced by T5 contain "contextual hallucinations", such as reading the idiom "shooting for the moon" literally, and that both extractive and abstractive models struggle on listicles. Ask, Assess, and Refine showed that on multi-entity list questions (QAMPARI), a model correcting itself without outside feedback drove correctness recall from 18.6 down to 4.8, while feedback from external metrics did not cause that collapse.

The closest prior designs differ in a specific way. Saved You A Click proposed a hybrid that classifies first and then routes simple hooks to extractive models and complex ones to abstractive models, but it did not build that hybrid. Our design does not route on a prior guess. A²R refines outputs using citation-based feedback but assumes the question format is known in advance; here the format is the unknown being inferred. Co-opNet reranks generated summaries with a factuality discriminator, and we borrow its generate-then-rerank pattern. Co-opNet's candidates, however, all share one output form. Our candidates differ in structure, and the reranker decides the structure. Billy-Batson won the type-classification task by pruning irrelevant paragraphs before classifying. It also notes that pruning can discard the paragraphs that carry multi-part spoilers. That is a constraint on our design: the verifier must see the full article, not a condensed one.

Core Claim

Two hypotheses follow from the argument above. First, selecting the spoiler type by the grounded support of shape-specific candidates should beat headline-driven classification on balanced accuracy, with the gain concentrated in multi-part recall. Second, at matched answer quality (measured by overlap with reference spoilers), this selection step should lower the rate of unsupported spoiler units compared with generating in a single predicted shape. A third, descriptive contribution is the first per-type audit of hallucination modes in clickbait spoiling. The audit asks whether type errors trigger predictable kinds of fabrication: invented list items after a false multi-part call, and over-compressed or entity-swapped phrases after a false phrase call.

Research Program

The candidate generator is a single instruction-tuned language model prompted three times per post. Each prompt asks for one shape, and the model is told to quote the article wherever possible. To generate better queries, headlines are first rewritten as explicit questions, following Jack-Ryder.

Each candidate is split into units: one unit for a phrase, one per sentence for a passage, one per item for a list. Every unit is aligned to the full article using an extractive question-answering model pretrained on SQuAD 2.0. Billy-Batson already uses this model for spoiler extraction. An entailment check then decides whether the aligned span supports the unit.

The support score for shape $t$ combines the fraction of supported units with a structural term. For multi-part candidates, the structural term asks whether the supported items are distinct and come from separate spans, and whether their number matches any count stated in the headline. This term is what keeps a list from passing by repeating one fact five times.

The final type is the one maximizing $\log p_{\text{clf}}(t) + \lambda \, s(t)$. Here $p_{\text{clf}}(t)$ is the probability a fine-tuned RoBERTa type classifier assigns to type $t$, $s(t)$ is the support score of that type's candidate, and $\lambda$ is a weight tuned on the validation split. If $\lambda = 0$, the system reduces to the standard pipeline. If the classifier term is removed, it becomes pure evidence-based selection. The chosen candidate is output with any unsupported units removed. If nothing is supported, the system abstains.

For the hallucination audit, two annotators label units on a stratified sample of WCS test outputs. The categories are supported, unsupported-fabricated, entity or role swapped, and figurative misreading, following the hallucination categories surveyed in Siren's Song in the AI Ocean. These human labels also measure how often the automatic verifier agrees with people.

Baselines

The do-nothing reference is the majority-class predictor for type, paired with a single prompted spoiler that has no type. We also include few-shot prompting for the type itself. Alexander Knox reports roughly 0.40 balanced accuracy for that approach with BLOOM, and we re-run it with our generator model so model size is held fixed.

The strongest type classifier is Billy-Batson's condensation-plus-RoBERTa-large system, which reports 74.14% balanced accuracy on the SemEval-2023 test set. This is the number to beat. The task's RoBERTa baseline reports 73.40% on validation. Hagen et al. report 80.39% balanced accuracy in their original setup, which is not the shared-task test split, so we cite it only as context.

For spoiler quality, the strongest reported result is BLEU-4 of 0.48 from Team John King's sequence-to-sequence model, as listed in the SemEval overview. We will also run a classify-then-generate pipeline with our own generator, so the routing comparison isolates the selection mechanism and not the choice of model.

All systems use the same 3,200/800/1,000 split, the same generator, and the same budget of validation tuning runs. We count beating Billy-Batson as a statistically significant gain in balanced accuracy, by paired bootstrap, together with a higher multi-part recall. We count a hallucination win as a lower unsupported-unit rate at BLEU-4 no worse than the classify-then-generate pipeline.

Two ablations isolate the key ingredient. The first keeps all three candidates but selects among them by the generator's own likelihood instead of grounded support. The second keeps the verifier but filters only the single candidate from the classifier's predicted type, without type re-selection. If the first ablation matches the full system, grounding is not what helps. If the second matches it, structural re-selection is not what helps.

Why not just…?

Why not just fine-tune a better type classifier on the headline and article? This is the most serious threat, because Billy-Batson's classifier already reads the article. The difference is that our method asks a different question. A classifier learns surface correlates of type. We test whether the article actually contains several distinct, separately supported items. Clark Kent's finding that annotators struggle on multi-part without seeing the spoiler suggests surface cues run out. If a stronger fine-tuned classifier closes the gap on multi-part recall, the second ablation will show it and the classification claim fails. In that case the hallucination-filtering claim would still be tested on its own.

Why not just use extractive question answering, which cannot invent text? Extraction does avoid fabrication, but single-span extractors cannot return scattered list items. The SemEval overview attributes the top BLEU-4 result to a generative model precisely because of multi-part cases. Our pipeline keeps extraction as the verifier rather than as the producer of answers. An extractive-only baseline is included, so the cost of giving up generation on each type is measured directly.

Why not just let the model critique and fix its own spoiler? A²R's QAMPARI result, where self-correction drove recall from 18.6 to 4.8, is direct evidence that self-critique fails on exactly the list-shaped outputs where multi-part spoilers live. We nonetheless run a self-refine baseline with the same generator and the same number of model calls, because the clickbait setting has not been tested.

Why not just rank article sentences by similarity to the headline? Jack-Ryder found that sorting sentences by Sentence-BERT similarity to the title lowered extraction performance, because the sentence that reveals the answer often shares little wording with the teaser. That finding is why our verifier aligns each unit to the article and does not score units by similarity to the headline.

Proposed Experiments

The data are the Webis Clickbait Spoiling Corpus 2022, as used in SemEval-2023 Task 5, and the Reddit SavedYouAClick and Facebook StopClickbait sets from Saved You A Click. The Reddit and Facebook data serve as a transfer test for noisier, more abstractive references. Metrics are balanced accuracy with per-class recall, BLEU-4 and BERTScore for spoilers, the share of correct end-to-end spoils as in Hagen et al., and the human-annotated unsupported-unit rate broken down by gold type.

The decisive check is an oracle-type experiment. We run the same generator and verifier with gold types, with Billy-Batson-style predicted types, and with evidence-based selection. Then we measure what fraction of the gap between predicted types and gold types the selection step closes. A large share of the gap closed, concentrated on multi-part posts, would support the proposal. A gap that persists while the hallucination rate still falls would mean grounding works as a filter but not as a type selector.

A second check is a cross-tabulation of type errors against hallucination category. It tests whether a false multi-part call predicts invented items above the base rate.

A third check compares condensed and full-article inputs to the generator. It tests whether Billy-Batson-style pruning increases fabrication, a question its authors raised but did not test.