When several people rate the same joke and disagree, humor datasets record the spread of their scores as "controversy." Models then try to predict that spread from the joke's text alone, and they do badly. This proposal tests a simple, testable alternative. Some of that disagreement may not come from who the raters are. It may come from the mood each rater was in when the joke reached them, and that mood is shaped by the images, jokes, and offensive items they saw just before. We would run a rating study that deliberately shifts raters' emotions partway through a session, using standard laboratory mood-induction procedures (brief exposure to validated negative, neutral, or amusing material). We would then fit a small recurrent model to each rater's sequence of ratings. A recurrent model is a network that carries a running internal summary, or "state," from one step to the next. The question is whether a latent mood state that rises and decays explains held-out ratings and item-level controversy better than fixed per-rater traits do.
The opportunity is concrete. On the SemEval-2021 humor benchmark, top systems detect intended humor at F1 near 0.98 but reach only about 0.63 on controversy, and simple threshold rules do about as well (SemEval 2021 Task 7: HaHackathon). If part of that variance is caused by session order and transient affect, then no text model can recover it. Datasets would then need to log presentation order and rater state, and the field's notion of "controversy" would need to separate a joke that divides people from a joke that divides moods.
Core Claim
We claim that induced negative affect does more than lower a rater's scores across the board. It changes how much a joke's content matters. Lab work shows that a negative emotional state weakens people's ability to filter irrelevant material out of working memory: under negative image primes, the brain-signal marker of working-memory load for "2 targets plus 2 distractors" stayed as high as for 4 targets (A negative emotional state impairs individuals' ability to filter distractors from working memory). Negative primes also flatten the normal brain response that favors self-relevant information (Negative Emotion Weakens the Degree of Self-Reference Effect). Getting a joke means holding and resolving an incongruity, so these findings motivate a specific prediction. Under negative affect, borderline jokes, meaning those with high offense ratings or dark targets, should lose more funniness than benign ones, and raters should diverge more on them.
This is a state-by-content interaction. A rater offset that drifts additively over the session cannot produce it, and a fixed rater embedding cannot produce it either. That is what makes the recurrent model more than decoration. Its gates let the current state change how item features map onto ratings. The closest experimental work, Humor Affects Fairness Considerations in the Gain and Loss Contexts, used pre-rated comics to induce positive mood before an Ultimatum Game. There, humor was the cause and economic choice was the outcome. We reverse the direction: mood is the manipulation and humor judgment, including its spread across raters, is the outcome. That paper also found humor priming had no effect in loss contexts (61.09% vs. 61.56% acceptance). This constrains our design: mild comic primes are fragile, so our negative induction must be strong enough to survive a rating task, and manipulation checks are mandatory.
Research Program
Stimuli. We draw items from two existing corpora. One is the HaHackathon set, 10,000 texts with humor ratings, offense ratings, and binary controversy labels from 20 raters (SemEval 2021 Task 7). The other is Humicroedit, about 15,000 micro-edited headlines rated by five judges, where the authors attribute modest agreement to political and personal differences (President Vows to Cut <Taxes> Hair). Using items with existing aggregate ratings lets us stratify a pool of roughly 600 items by prior funniness, prior offense, and prior controversy. Following Humor@IITK, we will not augment or paraphrase jokes, because that paper found such edits destroy the incongruity; the same paper found controversy is not a proxy for offensiveness, which we re-test.
Rating sessions. Each of about 300 online participants rates 150 items in a logged order. Every 25 items, a short block shows either negative images, neutral images, or amusing comics, modeled on the passive priming procedures in the cited work. Block order is counterbalanced across participants, and each item appears under each induction condition across the sample, so mood and item are crossed. Brief self-report probes of valence and arousal are collected at random points. Some probes enter the model; a held-out subset is reserved to check whether the learned state tracks reported mood. Item order within blocks is randomized per participant, so we can estimate carryover from preceding items independently of induction.
Models. Let $r_{j,t}$ be rater $j$'s rating of the item shown at position $t$, $x_t$ that item's text features (a frozen sentence encoder plus its prior offense and funniness scores), and $m_t$ an indicator of the most recent induction block. The linear reference model keeps a scalar mood that decays exponentially:
$$s_{j,t} = \lambda s_{j,t-1} + a^\top m_t + b\, r_{j,t-1}, \qquad \hat r_{j,t} = \mu(x_t) + u_j + s_{j,t-1},$$
Here $\lambda$ is a decay rate, $u_j$ a fixed rater offset, and $\mu(x_t)$ the item's predicted mean. The gated model replaces this update with a small gated recurrent unit (GRU, a recurrent cell with learned gates deciding what to keep and overwrite) over $(x_t, m_t, r_{j,t-1})$. Its prediction head takes the product of state and item features, so the state can rescale the influence of offensiveness. Both models output a full rating distribution and are trained with the distributional "soft" loss of SemEval-2021 Task 12: Learning with Disagreements.
Short memory is acceptable here. Do RNN and LSTM have Long Memory? proves that standard gated recurrent networks forget at an exponential rate, and induced moods are known to fade quickly. We treat exponential decay as a hypothesis, not a flaw. If the data show slower, power-law carryover across blocks, we would add that paper's fractional-memory variant as an ablation. For interpretation we borrow from NeuralHydrology. That work showed individual LSTM cell states correlate with unobserved physical stores such as snowpack. We ask the analogous question of whether our state correlates with held-out mood reports.
Baselines
The naive reference point predicts each rating as the item's mean rating from the original corpus, with no rater or order information. The traditional step up is a rater-offset model: item mean plus a per-rater bias, which is what majority-vote or mean aggregation implicitly assumes away. On aggregated distributions, the strongest existing method for humor disagreement is Gaussian Process Preference Learning. It remained the best system on the Le-wi-Di Humour data, where adding crowd layers to BERT hurt (SemEval-2021 Task 12). We found no reported numeric score on Lacuna for it, so we re-run it ourselves on our data.
For per-rater modeling, the strongest competitor is the demographic mixture-of-experts of Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives. It routes items through experts chosen by annotator demographics and beat jury-style baselines on high-disagreement data. We implement it with demographics collected at intake and add learned rater embeddings, giving it every static advantage. For text-only controversy, the reference numbers are the HaHackathon controversy F1 of about 0.63 for top systems (Meaney et al.), 0.4764 for DeBERTa-large (HumorHunter), and 0.5024 for an SVM (RedwoodNLP). We fine-tune a DeBERTa model with soft labels on our items to give a comparable in-house figure.
All learned baselines share the frozen text encoder, the same splits, and the same tuning budget of 50 random-search trials per model. Splits hold out entire raters, so we test generalization to new people given their first 25 ratings, not memorization of known raters. The key ablations are the linear decaying-state model, which isolates whether gating and the state-by-content interaction matter beyond a running offset, and the gated model with $m_t$ removed, which isolates the contribution of knowing the induction condition. Beating the static baselines means a reliable reduction in held-out negative log-likelihood per rating, with confidence intervals from bootstrapping over raters that exclude zero. Beating the linear model means the same, concentrated on high-offense items as the core claim predicts.
Why not just…?
Why not just fine-tune a transformer on the text with soft labels? It cannot see the order in which a rater met the items or what they saw before. If our hypothesis holds, part of the variance is therefore invisible to it by construction. The HaHackathon results, where DeBERTa reached 0.4764 controversy F1 while detecting humor at 0.948 (HumorHunter), are consistent with this ceiling but do not prove it. Our design tests it directly. Because each item is rated under every induction condition, any condition-dependent shift in an item's rating distribution is variance a text model must miss. We report its size.
Why not just use static rater embeddings or demographic experts? This is a real competitor. Most rater variance may well be stable taste, as the race-by-topic interactions in Reconsidering Annotator Disagreement about Racist Language suggest. Within-rater crossing of mood and item settles it. If the static model with rich rater embeddings matches the stateful model on held-out sessions, the mood hypothesis is wrong for humor ratings, and we will report that as the result.
Why not just use a linear exponential-decay correction, or a mixed-effects regression with a lagged rating? This is the most serious threat. Classic order and contrast effects may be fully captured by a decaying additive offset, which is exactly a one-unit linear recurrent model. We therefore make that model the primary ablation and do not argue around it. The gated model is justified only if it wins specifically through the interaction with offensiveness. If it does not, the paper's contribution narrows to measuring induced-mood and carryover effects on humor ratings with a linear state model. That is still a new empirical finding, with a cheap correction others can apply.
Why not just randomize presentation order and let the effects average out? Randomization removes bias in item means, but it adds variance to individual ratings. Variance across raters is precisely what controversy labels measure. Our counterfactual analysis below quantifies how much of the measured controversy randomization manufactures.
Proposed Experiments
The first check is behavioral and needs no model. A mixed-effects analysis of ratings tests whether negative induction lowers funniness, confirmed by self-report manipulation checks, and whether the drop is larger for items with high prior offense. That interaction is the hypothesis's central prediction. The analysis also tests whether induced negative affect raises between-rater variance on those items.
The second check compares held-out per-rating likelihood across the naive, rater-offset, Gaussian process, demographic-expert, linear-state, and gated-state models on unseen raters. We break results down by item offense level and by distance from the last induction block, which shows whether advantages decay as fast as induced moods should.
The third check is interpretive. Correlating the gated model's state with the held-out valence probes shows whether the state tracks reported mood, as LSTM cell states tracked hidden water stores in hydrology.
The fourth check is counterfactual. Using the fitted model, we simulate re-rating each item by the same raters in a mood-neutral state. We then recompute HaHackathon-style controversy labels, where variance above the median counts as controversial, and report what fraction of items change label. If that fraction is substantial, controversy as currently defined partly measures session conditions rather than the joke. If it is near zero, text-intrinsic and trait-based accounts of humor disagreement stand, with a stronger justification than before.
References
- SemEval 2021 Task 7: HaHackathon, Detecting and Rating Humor and Offense
- A negative emotional state impairs individuals’ ability to filter distractors from working memory: an ERP study
- Negative Emotion Weakens the Degree of Self-Reference Effect: Evidence from ERPs
- Humor Affects Fairness Considerations in the Gain and Loss Contexts
- “President Vows to Cut <Taxes> Hair”: Dataset and Analysis of Creative Text Editing for Humorous Headlines
- Humor@IITK at SemEval-2021 Task 7: Large Language Models for Quantifying Humor and Offensiveness
- SemEval-2021 Task 12: Learning with Disagreements
- Do RNN and LSTM have Long Memory?
- NeuralHydrology - Interpreting LSTMs in Hydrology
- Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives
- HumorHunter at SemEval-2021 Task 7: Humor and Offense Recognition with Disentangled Attention
- RedwoodNLP at SemEval-2021 Task 7: Ensembled Pretrained and Lightweight Models for Humor Detection
- Reconsidering Annotator Disagreement about Racist Language: Noise or Signal?