Sarcasm as a Violation of Thread Sentiment Flow

Sarcasm detectors learn to spot sarcasm from surface cues: particular phrases, emoji, and the slang of whatever year their training data came from. Those cues age quickly. We propose a detector that asks a different and more durable question: given how the sentiment of a conversation has been moving, is this reply's literal tone the opposite of what the thread predicts? A reply that says "great, love it" in a thread that has grown steadily angrier is suspicious whatever words it uses. The mechanism has three parts. A sentiment scorer driven by natural-language label descriptions reads each turn's surface polarity and needs no sarcasm labels. A sequence model of how sentiment normally flows through conversation threads is trained without sarcasm labels. A small classifier then sees only how far the reply departs from the predicted flow. The thread model can be refreshed on unlabeled new conversations, so the detector can follow language change without new sarcasm annotation.

The claim we test is specific and falsifiable. On Reddit threads with author-marked sarcasm spanning 2009–2017, a detector that must route its decision through this "flow violation" signal should lose less accuracy when tested on years after its training data than a fine-tuned transformer given the same thread context. We expect it to lose to that transformer inside its own training era. If it does not degrade more slowly, the idea is wrong and we say so.

Why this intersection, and what is actually open

Sequential sentiment modeling rests on context coherency: sentiment tends to persist across neighboring sentences unless a discourse marker breaks it. Several methods exploit this. Posterior regularization propagates confident sentence labels to ambiguous neighbors, and isotonic CRFs constrain how sentiment shifts from sentence to sentence. Sentiment Flow reduces reviews to sequences like [pos, pos, neg, pos]. It finds that these skeletons transfer across domains far better than words do, with 60.9% hotel-to-movie accuracy against 39.8% for bag-of-words. That last result motivates us: abstract sentiment trajectories survive a change of vocabulary. All of this work treats sarcasm as noise that breaks coherency. We treat that breakage as the signal. None of it studies multi-turn dialogue or change over time.

Sarcasm work has already shown that context matters. On SARC, 1.3M self-annotated /s Reddit comments with full threads, human accuracy rises from 82% to 92% when the thread is shown. Transformer models that concatenate conversational turns (BERT with conversation context; a contextual Twitter transformer) use this context without structuring it. Author-context modeling detects deviation from a user's usual sentiment baseline. That is the closest idea in spirit to ours, but its baseline is the author's history, where ours is the thread's trajectory. SIGN builds on the view that sarcasm is sentiment inversion and translates single tweets into literal ones, without using context. iSarcasm constrains how much surface cues can achieve. Models trained on hashtag labels fall from 0.874 to 0.364 F-score on author-intended sarcasm.

Temporal drift work quantifies the decay. TimeLMs reports nearly a 10% drop after one year of future Twitter data and counters it with quarterly continued pretraining. Trend-based retraining for NER recovers 2.70–5.71 F1 by selecting bursty n-grams for retraining. That paper admits frequency heuristics miss meaning shifts that come without frequency spikes, which is exactly the kind of drift sarcasm tropes undergo. TweetEval found that irony was the one task where a Twitter-from-scratch model beat continued pretraining, which suggests irony is unusually tied to platform culture. All three work on isolated tweets. None evaluates conversational sarcasm on chronological splits, even though SARC's eight-year span makes that possible.

Semantic label embedding supplies the literal-sentiment scorer. Classification by matching text to natural-language label descriptions (Gen-Z, 91.7 F1 zero-shot on SST2 against 87.1 for the best discriminative zero-shot baseline; label definitions for SRL, with gains out of domain and on unseen predicates) gives a sentiment reader that needs no in-domain labels. Its tendency to read surface polarity is exactly what we want for the literal tone of a sarcastic reply. The SRL paper also warns about cost, a 3.1× slowdown, and about generic definitions carrying little signal. Both shape our design.

Core Claim

A sarcasm decision factored through thread sentiment flow depends on pragmatic structure rather than era-specific vocabulary. It should therefore degrade more slowly under temporal drift. Because its only learned sequence component is trained without sarcasm labels, it can also be adapted to a new era using unlabeled threads alone.

Research Program

The first component is a literal sentiment scorer. It assigns each turn a distribution over negative, neutral, and positive using Gen-Z-style scoring against several paraphrased, Reddit-contextualized descriptions of each polarity. The descriptions say what the words express on the surface, not what the author means. Each turn is scored once and cached, so a thread of $n$ turns costs $3n$ description matches. That keeps the SRL-style slowdown bounded and shared across every reply in the thread. We validate that the scorer reads surface tone on iSarcasm. For author-verified sarcastic tweets, the score of the tweet and the score of the author's own literal rephrase should usually have opposite signs.

The second component is a thread flow model: a small recurrent or transformer model over the sequence of per-turn sentiment distributions plus coarse structure features such as depth and whether the author has changed. It is trained to predict the literal sentiment of the next turn. Training uses replies not marked /s, which are plentiful and need no human labels. Unmarked sarcasm remains as label noise, and we measure its effect below. This is sequential sentiment modeling moved from monologue reviews to dialogue. Unlike the stationary transition matrices of CRF and Markov approaches, it is retrained per era.

The third component is the decision bottleneck. For a reply $r$ in thread $h$, the violation score is

$$v(r,h) = s_{\text{lit}}(r) - \mathbb{E}_{\text{flow}}[\,s \mid h\,],$$

where $s_{\text{lit}}(r)$ is the scorer's expected polarity of the reply on a $[-1,1]$ scale and the second term is the flow model's predicted polarity for the next turn. A small classifier sees $v$, the flow model's predictive uncertainty, and the reduced flow sequence. The sequence is compressed with Sentiment Flow's Change operation, which collapses runs of identical adjacent states. The classifier does not see the reply's text. This bottleneck is the key ingredient: it removes the lexical shortcut channel. We also build a hybrid that adds these features to a text cross-encoder. The hybrid tests whether the flow signal adds robustness even when text is also available.

For temporal adaptation, when moving to a new year we retrain only the flow model on that year's unlabeled non-/s threads. The sarcasm classifier, trained on the old era, stays frozen. This is the protocol that distinguishes our proposal from TimeLMs-style adaptation, which refreshes the encoder but keeps a fixed, era-specific decision head.

Baselines

The naive reference points are the majority class and SARC's own bag-of-bigrams classifier. For bag-of-bigrams, SARC reports 75–76% accuracy on the balanced pairing task, where the model picks which of two replies is sarcastic, and 25–27% F1 in the natural unbalanced setting. Both were measured on random splits, so we rerun them on our chronological splits rather than quote them as targets.

The strongest competitor is a RoBERTa cross-encoder fine-tuned on the reply concatenated with its full thread, following the conversation-context transformers cited above. It sees strictly more information than our bottleneck and can in principle learn incongruity implicitly, so it is the model to beat on robustness. No reported result was found on Lacuna for any context-aware transformer on SARC under chronological splits. We run it with the same training years, the same thread truncation, and the same hyperparameter search budget in GPU-hours as our system, which includes the flow model's training cost.

The second strong competitor gives the cross-encoder era-matched continued pretraining on Reddit text from each test year, in the manner of TimeLMs. This uses the same unlabeled data our flow model gets, so the comparison asks where that unlabeled data is best spent. TimeLMs reports beating BERTweet on TweetEval irony, but no specific number is in the file. No reported result was found on Lacuna for this setup on SARC.

The third competitor is zero-shot Gen-Z with a description of the sarcastic label, conditioned on the thread. No reported result was found on Lacuna for Gen-Z on any sarcasm benchmark. Its reported numbers are SST2 sentiment and do not transfer.

Ablations isolate each ingredient. A single-step variant replaces the flow model with the parent comment's literal sentiment. It tests whether trajectory matters beyond pairwise incongruity. A frozen-flow variant skips per-era retraining, which tests the adaptation claim. A lexicon-scorer variant replaces the description-based sentiment scorer with a sentiment lexicon, which tests the label-description component.

Beating a competitor means two things. The slope of accuracy against years since training must be significantly flatter across bootstrap resamples of test threads. Accuracy in the most distant test year must match or exceed the competitor, even if in-era accuracy is lower.

Why not just…?

Why not just fine-tune a transformer on the whole thread? This is the real threat. It may learn flow incongruity internally, and nothing in the file shows it degrades faster than a structured model on conversational data. The file shows only that raw lexical representations transfer poorly across review domains (Sentiment Flow) and that irony is tightly coupled to platform culture (TweetEval). The head-to-head degradation curves settle it. If the cross-encoder's slope is as flat as the bottleneck's, the central claim fails. The hybrid then tells us whether flow features have any residual value.

Why not just keep the encoder current with continued pretraining? TimeLMs shows that this slows spoilage on single tweets. However, it refreshes representations while leaving a decision boundary learned on old-era labels. Our adaptation instead refreshes a model of conversational expectation. The two compete for the same unlabeled data, so the second baseline compares them directly, and we also test the combination. If continued pretraining alone closes the gap, the adaptive flow model is unnecessary and we will report that.

Why not just contrast the reply with its parent comment? Pairwise incongruity is simpler and may capture most of the signal, since many sarcastic replies answer one comment. The single-step ablation measures this directly. We keep the full trajectory model only if it improves future-year accuracy, especially on deeper threads, where the class-sequential-rule finding that later states weigh more (CSR) suggests trajectory information matters.

Proposed Experiments

On SARC we build year-level chronological splits. We train on 2009–2012, validate on 2012 data held out by thread, and test separately on each of 2013 through 2017. Metrics are sarcastic-class F1 at the natural imbalance, which is the realistic regime where baselines collapse to 25–27%, pairing accuracy, and AUROC. Pairing accuracy and AUROC are insensitive to base rate, which matters because /s usage conventions themselves may change over time. We report per-year /s rates, and we stratify by subreddit so that drift is not confused with a changing topic mix.

The decisive check is the degradation-slope comparison between the bottleneck, the cross-encoder, and the TimeLMs-style cross-encoder. We state the expectation in advance: the bottleneck trails in-era and its slope is flatter.

Two further checks probe the mechanism. First, we measure contamination from unmarked sarcasm by retraining the flow model with an increasing fraction of /s replies deliberately included. If the violation signal is fragile to such noise, the gap should shrink as the fraction rises. Second, on iSarcasm, which has no threads, we test only the literal-scorer premise through the polarity flip between each tweet and its author's rephrase. We also test whether the reply-only component generalizes to intended rather than perceived sarcasm better than a hashtag-trained model, whose F-score of 0.364 on iSarcasm is the reported reference.

Finally, we inspect errors at the far end of the timeline. For sarcastic replies the cross-encoder misses in 2017 but the bottleneck catches, we check whether they contain vocabulary absent from the training years. That would be direct evidence that the flow signal carries detection across vocabulary change.