Taxonomy Backoff for Situational Sarcasm

Much sarcasm contains no internal contradiction at all. "Love spending my birthday in the emergency room" is sarcastic only because readers know that emergency rooms are bad places to be. A detector therefore needs a prior on how good or bad the described situation typically is, and it needs that prior most for rare situations. Those are the ones training data never covers, and they are also where language models are most likely to invent an answer. We propose to estimate these situation priors from aggregate human evidence: what people write, search for, and do when they encounter a situation. That evidence is sparse in the tail, so we pool it up an is-a concept hierarchy (a taxonomy such as WordNet, where "emergency room" sits under "hospital" and then "medical facility"). The model climbs only as far as the evidence requires and stops before the abstraction erases the specific meaning. The sarcasm detector then reads the gap between the sentiment the speaker expresses and this situation prior, together with how confident the prior is.

The two-sentence version is this. Sarcasm about situations depends on knowing whether the situation is normally bad. We estimate that from human reaction evidence, and for rare situations we borrow evidence from parent concepts in a taxonomy, weighting each level by how much evidence it actually has. The new ingredient is the evidence-counted backoff. Prior sarcasm systems inject commonsense as flat, unweighted replacements. Prior taxonomy methods pick an abstraction level by hand-set thresholds, and none of them has been applied to figurative language.

Core Claim

Implicit incongruity is the case where positive words describe an objectively negative situation. Joshi et al. named it as the main failure of feature-based detection: their system broke whenever the contrast hinged on unstated world knowledge (Harnessing Context Incongruity for Sarcasm Detection). Current knowledge-enhanced detectors fill that gap with flat knowledge. CSDGCN uses a generative commonsense model (COMET, a transformer trained to produce likely intents and reactions for an event) to replace dependency-tree nodes with emotion words such as "traffic jam" becoming "irritated" (Commonsense Knowledge Enhanced Sentiment Dependency Graph for Sarcasm Detection). EICR retrieves web passages with a hybrid BERT-plus-BM25 retriever, uses them to condition an LLM that writes background commonsense, and then reasons over an incongruity graph (Detecting Emotional Incongruity of Sarcasm by Commonsense Reasoning). Neither system knows how much evidence stands behind a given piece of knowledge, and neither can generalize a rare situation through its category.

We claim that a situation-valence prior which is explicitly evidence-weighted and pooled over a taxonomy will improve detection most on situations with little direct evidence. We further claim that this prior can be checked against author-written literal rephrasals, independently of any sarcasm classifier.

The Estimator

Let $s$ be a situation phrase extracted from an utterance, for example the verb phrase and argument "spend birthday in emergency room", and let $c$ be its node in the taxonomy. The node for $s$ is chosen with the contextualized conceptualization approach of CAT over Probase and WordNet (CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoning), which we reuse as the linking step only. Let $n_c$ be the number of evidence items attached to node $c$, and let $\bar v_c$ be their mean valence on a scale from $-1$ (bad) to $+1$ (good). The smoothed prior is defined recursively from the root down:

$$\hat v(c) = \frac{n_c\,\bar v_c + \kappa\,\hat v(\mathrm{parent}(c))}{n_c + \kappa},$$

where $\kappa$ is a single pooling strength fitted by maximizing held-out likelihood of node valences. This is standard hierarchical shrinkage. A well-attested node keeps its own estimate. A node with no evidence inherits its parent's estimate.

The same formula handles the "loss of contextual flavor" problem that CAT itself flags, where abstracting "watching a movie" to "observing" strips away the entertainment. Pooling up a level is only harmful when the child differs from its parent, and in exactly that case the child's own evidence dominates the estimate once it has any mass. An uncertainty term, the posterior variance, which shrinks roughly as $1/(n_c+\kappa)$ as in a conjugate normal model, travels with $\hat v$.

The detector receives three quantities: the expressed sentiment of the utterance $e$, the prior $\hat v(c)$, and its variance. Its incongruity feature is $e - \hat v(c)$, gated by that variance. Low variance means the prior is trusted, so a large gap counts heavily; high variance means the gap is discounted.

Where the Evidence Comes From

The behavioral semantic mapping literature grounds short, ambiguous text in what people do rather than in which words co-occur. Examples include clicked destinations and session reformulations distilled into query embeddings (Search Query Embeddings via User-behavior-driven Contrastive Learning), and the retriever treated as an environment whose query-document and document-document overlaps reveal intent (Query Variant Detection Using Retriever as Environment). We reuse the second idea for reproducibility.

Each situation phrase is issued as a query to a fixed retriever over a large corpus of human-written text. The evidence items are the top retrieved documents. Each document is scored for the writer's valence toward the situation, and separately for whether it is remedial: complaints, refund requests, and coping advice count as strong negative signals, because people rarely seek remedies for good things. Where a query log is available, reformulation paths from the situation to remedial intents ("X" followed by "X refund" within a session, mined with the same 300-second window UBIQUE uses) add a second evidence channel.

This field's known weaknesses shape the design. Click logs fail on tail queries with no history, which is exactly what the taxonomy backoff addresses. Logs also carry popularity bias, which is why each evidence item is counted rather than trusted, and why a node dominated by a few duplicate sources receives a reduced effective $n_c$. The two evidence channels differ in kind. Retrieval evidence reflects what people write about a situation, while log evidence reflects what they do about it. The program tests whether the second adds anything over the first rather than assuming it does.

Research Program

The first stage builds the situation lexicon and prior. We extract situation phrases from the training and test sets of the sarcasm benchmarks, link them to taxonomy nodes, attach retrieval evidence, and fit $\kappa$.

The second stage validates the prior on its own, before any sarcasm model is involved. iSarcasm has authors supply a non-sarcastic rephrasal of each sarcastic tweet (iSarcasm: A Dataset of Intended Sarcasm). That rephrasal states the author's true attitude toward the situation, so the sentiment of the rephrasal gives a free, author-sourced label for situation valence. We measure how well each prior source predicts it, bucketed by $n_c$ of the linked node.

The third stage adds the gated incongruity feature to a fixed detector and measures gains stratified by situation rarity. The fourth stage runs the ablations and the evidence-source comparisons described below.

Baselines

The naive reference point is a fine-tuned RoBERTa-large classifier on the utterance text alone, with no external knowledge. It shows how much any knowledge adds. A second weak reference is the same classifier given the prior conversational turn where the dataset provides one, following Sarcasm Analysis Using Conversation Context. That work reports 73.32% F1 on IAC-V2 with a conditional LSTM, and it found that prior context helps while sarcasm grounded in outside facts is still missed.

The strongest existing method is EICR, because it already combines retrieval-grounded commonsense with explicit incongruity reasoning, and it reports beating GPT-4o prompting. Its reported results are 70.4% Macro-F1 (83.3% accuracy) on iSarcasm, 80.3% Macro-F1 on SemEval-2018 Task 3, 84.3% Macro-F1 on Ghosh, and 83.8% Macro-F1 on IAC-V2 (Detecting Emotional Incongruity of Sarcasm by Commonsense Reasoning). On the Riloff dataset, whose construction centers on positive sentiment paired with a negative situation, the relevant reported number is CSDGCN's 87.16% accuracy (Commonsense Knowledge Enhanced Sentiment Dependency Graph for Sarcasm Detection). We found no reported result on Lacuna for EICR on Riloff.

We will rerun both systems ourselves on identical splits, with the same backbone size, the same number of hyperparameter trials, and the same retrieval corpus for EICR's retrieval stage. Beating EICR means a statistically significant Macro-F1 gain across five seeds on the rare-situation subset, the tweets whose linked node has $n_c$ below a preset threshold, with no loss on the frequent subset. Matching overall numbers while winning only on the tail would still support the claim. Winning overall while failing to win on the tail would not.

Two ablations isolate the key ingredient. The first uses flat priors, $\kappa=0$, so every node stands alone and nodes without evidence get a neutral prior; this corresponds to CSDGCN-style flat knowledge. The second uses fixed-height hypernymization, where every situation is mapped exactly $k$ levels up, as in Conceptual Captions' replacement of rare names by categories (Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning), with $k$ tuned on development data. If fixed-height backoff matches evidence-weighted backoff, the adaptive component is unnecessary and we will say so. A further ablation removes the variance gate to test whether confidence, not only the prior, matters.

Why not just…?

Why not just ask an LLM whether the situation is normally good or bad? This is the most serious threat, and it may well work for common situations. What the file offers is indirect. EICR beat zero-shot GPT-4o at whole-utterance sarcasm classification. The file also documents that generative models hallucinate implicit commonsense when prompted. Neither result shows that an LLM is a poor source of situation valence specifically. We therefore treat LLM-elicited valence as a competing evidence source placed inside the same estimator. In one variant it replaces retrieval evidence entirely. In another it serves as an extra evidence item per node. The decisive measurement is agreement with iSarcasm author rephrasals, bucketed by rarity. If LLM valence matches retrieval-plus-backoff on the rare buckets, the behavioral component adds nothing. The contribution then shrinks to "hierarchically calibrated LLM priors", and we would report it that way.

Why not just fine-tune a larger encoder, or add conversation history? Oprea and Magdy found that models trained on hashtag-labeled data fall from F-scores up to 0.874 to at most 0.364 on author-intended sarcasm, which suggests the models fit surface cues rather than situations. Ghosh et al. found that conversational context does not recover sarcasm grounded in outside facts. Our text-only and prior-turn baselines, run at matched scale, quantify how much of the tail gap remains after both remedies.

Why not just keep EICR's retrieved passages and let the model sort it out? EICR already retrieves, but it retrieves for the whole utterance and passes free text on to an LLM. It keeps no count of evidence per concept, and it cannot pool across related concepts when retrieval returns little. The flat-prior ablation plays this role in controlled form. If EICR's retrieval already saturates the tail, we will see it as no gain over EICR on the rare-situation subset.

Proposed Experiments

The benchmarks are Riloff, Ghosh, iSarcasm, IAC-V2, and SemEval-2018 Task 3. The SemEval dataset separates verbal from situational irony, which gives a natural test of whether gains concentrate on situational items. The metrics are Macro-F1 and accuracy, reported overall and on rarity-stratified subsets defined before training.

The first decisive check is prior validity. The smoothed prior must predict author-rephrasal sentiment on iSarcasm better than the flat prior in the low-evidence buckets, and no worse in the high-evidence buckets. If backoff does not help here, the detector experiments are moot.

The second check is that detection gains must be largest where $n_c$ is smallest. A uniform gain would point to a generic extra feature, not tail generalization.

The third check probes over-generalization directly. We will build a small, carefully annotated contrast set of child-parent pairs whose valences diverge, such as "surprise party" under "surprise", and require that the smoothed prior keep the child's valence while fixed-height backoff flips it.

The fourth check compares the retrieval channel with the LLM channel and, where logs are obtainable, with the reformulation channel. This settles whether evidence from human behavior earns its cost.