Reading Sarcasm as Stance in Persuasion Dialogue

Models that predict whether an argument changes someone's mind read every reply literally. When a challenger on Reddit's ChangeMyView forum writes "Great point, because nobody has ever been hurt by a ladder," a literal model sees agreement and positive sentiment. A human reader sees mockery of a specific claim from the previous turn. This proposal asks how much of the remaining error in persuasion prediction comes from that misreading. It also tests a fix. Instead of tagging each turn with how positive or negative its words sound, we tag it with its intended stance toward the specific earlier claim it targets: agree or disagree, and whether that stance is expressed literally or through inversion.

The bet has two parts. First, sarcasm in argument is mostly targeted inversion of an opponent's words, so a representation keyed to "which claim, and for or against" captures it better than a general sarcasm score. Second, stance toward a target means the same thing whether the topic is guns or opera. Word-level sentiment does not: "unpredictable" is praise for a thriller and a complaint about a car. A stance-based representation should therefore also narrow the cross-topic drop that persuasion models show.

Core Claim

Current sequential persuasion models are blind to inversion. The Attentive Interaction Model, which aligns challenger sentences with "vulnerable" sentences in the original post (Attentive Interaction Model), has this blind spot. So does the Dynamic Topic-Discourse Memory Network, which separates topic flow from discourse markers across turns (What Changed Your Mind). So does the proposition-sequence model for Debate.org (Exploring the Role of Argument Structure in Online Debate Persuasion). All three treat surface wording as a literal proposition, and none models non-literal moves.

We claim this blindness is measurable with a controlled test. Rewrite a reply from literal to sarcastic while keeping its intended stance fixed, and an inversion-blind model's prediction moves. We further claim that replacing surface sentiment with target-relative intended stance reduces both that sensitivity and the drop on held-out topics. The novelty is the stance-relative unit and the counterfactual test, not another sarcasm feature.

The closest prior work does not answer this question. SkOTaPA studies sarcasm, sentiment and resistance after persuasion attempts, but it scores isolated Instagram comments with no sequence. It finds that a sarcasm model used as a stand-in for skepticism reaches only 0.15 F1, and negative sentiment only 0.48. We read that result as a constraint: a generic sarcasm or sentiment score is the wrong unit for resistance, which is why we model stance toward a target instead. Rhetorical Device-Aware Sarcasm Detection supplies two ingredients. One is its nine-device taxonomy, where echoic mention (repeating someone's words in order to mock them) is the device we expect to dominate in debate. The other is its method of building minimal pairs, sarcastic and literal versions of the same dialogue, with a large language model (LLM). That work stays in sitcom banter and never connects devices to belief change.

Research Program

The first component is an annotated diagnostic set drawn from ChangeMyView threads. For roughly two thousand challenger replies and responses by the original poster, annotators mark four things: the target span in the preceding turn, the surface polarity, the intended stance toward that target (agree, disagree, or neutral), and whether the stance is inverted, with the device from the nine-way taxonomy. We do not trust third-party judgments of sarcasm without checks, because iSarcasm shows that third-party annotators miss 30% of intended sarcasm and mislabel 45% of non-sarcastic tweets. Annotation therefore uses the thread's own evidence: the original poster's reply and quoted rebuttals. We report agreement separately on inverted and literal items.

The second component is a stance tagger. It takes a pair of turns and outputs the target span and the intended stance. It is trained on the annotations, on minimal pairs built in the style of Rhetorical Device-Aware Sarcasm Detection from ChangeMyView replies, and on a weak signal from echoic overlap: lexical repetition of the previous turn combined with flipped surface polarity.

The third component replaces the turn-level sentiment and discourse inputs of existing sequence models with the tagged stance trajectory. Concretely, each turn pair becomes a token that records the target, the stance and whether it is inverted. These tokens feed into the arrangement encoders of Unveiling the Power of Argument Arrangement, a BiLSTM over clustered argument-unit sequences, and into the dynamic memory model.

Two analyses address tensions in the research file. The first asks whether sarcastic challenges lower the chance of a delta (the original poster's award for a changed view). That would be the debate analogue of the "strategy sabotage" effect in Examining the Ordering of Rhetorical Strategies, where attention to overt strategy correlates at R = −0.90 with request success. The second asks whether the effect depends on position in the thread: an opening jab versus a closing refutation.

Baselines

The naive reference points are the logistic regression baseline from the Attentive Interaction Model paper, which reports 70.9% AUC (area under the ROC curve) on in-domain ChangeMyView, and a majority-class predictor. We re-run both on our splits. The strongest structured competitor for delta prediction is the full Attentive Interaction Model at a reported 72.0% AUC (Attentive Interaction Model). On pairwise path ranking, it is the Dynamic Topic-Discourse Memory Network at a reported 75.1% accuracy (What Changed Your Mind). For arrangement-based prediction, BERT with Sequence Graph Transform embeddings reports 0.85 macro-F1 on balanced dialogues (Unveiling the Power of Argument Arrangement). On Debate.org the hybrid model reports 77.28% accuracy (Exploring the Role of Argument Structure).

We consider the strongest unstructured competitor to be a long-context encoder fine-tuned end to end on full threads. That choice follows SkOTaPA, where a fine-tuned RoBERTa-large beat prompted GPT-3.5-turbo. We also include a prompted LLM.

Every system gets the same train, dev and test threads, the same topic-held-out split, and an equal hyperparameter search budget measured in trials. Model size matches across systems wherever architectures allow. Three ablations isolate the key ingredient. The first adds an off-the-shelf sarcasm probability per turn. The second adds a surface sentiment trajectory. The third keeps our stance tags but removes the inversion flag and the target span.

We count the proposal as beating the field if it does two things on the topic-held-out split. It must beat the best of these competitors on AUC by more than the seed-to-seed variance, and it must cut counterfactual instability, defined below, by at least half relative to that competitor. In-domain parity combined with clear gains on those two measures also counts as a positive result.

Why not just…?

Why not just fine-tune a large encoder end to end on whole threads and let it learn sarcasm implicitly? This is the real threat, and the in-domain numbers may not separate the two approaches. Two experiments settle it. The counterfactual inversion test shows whether the end-to-end model's predictions shift when a reply is made sarcastic but its stance is held fixed. The topic-held-out split matters because Attentive Interaction Model reports that interaction models fall back to lexical, TF-IDF-like cues on unseen topics. If the end-to-end model is stable under inversion and transfers across topics as well as ours does, we will report that the stance representation is unnecessary.

Why not just add a sarcasm detector's score as a feature? The evidence says such detectors do not track intended sarcasm. State-of-the-art models score 0.874 F-score on hashtag-labeled data but only 0.364 on author-verified intended sarcasm (iSarcasm). A sarcasm model used as a stand-in for resistance reaches only 0.15 F1 in SkOTaPA. A detector score also omits what was mocked and whether the mockery agrees or disagrees with it. The ablation that adds only a sarcasm probability tests this directly.

Why not just rewrite sarcastic turns into literal text with an LLM, then run an existing persuasion model? This preprocessing is plausible. Sarcasm SIGN shows that literalizing sarcasm is feasible, though it reached only 46.2% intended-sentiment accuracy, and it fails when the sarcasm has no explicit sentiment words. Rewriting also tends to copy the surface wording rather than flip it, which reintroduces the problem. Because this alternative is plausible, we treat it as a second implementation of the same claim, not as a rival. We run a "literalize-then-predict" pipeline under the same test. If it matches the stance tagger, the conclusion is that removing inversion matters and the particular representation does not.

Why not just prompt a large LLM with the whole thread? On structured persuasion prediction, a fine-tuned Llama-2-7B reached 0.45 macro-F1 against 0.85 for an arrangement model (Unveiling the Power of Argument Arrangement). (RSA)² documents a literal bias in base LLMs (listener probability 0.73 versus 0.85 with modeled rhetorical strategies). We still include a prompted baseline under the same inversion test.

Proposed Experiments

The first check is the counterfactual inversion test. Using the minimal-pair method of Rhetorical Device-Aware Sarcasm Detection, we produce sarcastic rewrites of literal-disagreement challenger replies on the ChangeMyView test split, plus literal rewrites of sarcastic ones. Humans verify that intended stance is preserved. The metric is counterfactual instability: the mean absolute change in each model's predicted delta probability across a pair. Our prediction is that literal models shift substantially and the stance-based model barely moves.

The second check repeats delta AUC on the Attentive Interaction Model's split, with an added topic-held-out split. It also repeats pairwise accuracy on the Dynamic Topic-Discourse Memory Network's ChangeMyView paths and accuracy on Debate.org. Results are broken out by whether the thread contains an annotated inverted turn. If errors concentrate in inverted threads and our gains appear there, the core claim holds. If gains appear evenly across threads, the improvement comes from something other than inversion.

The third check evaluates the stance tagger itself, reporting macro-F1 per device on the diagnostic set. As a sanity check it is also run on iSarcasm, where the reported maximum F-score is 0.364. We expect echoic mention to be the most frequent device and innuendo the hardest.

The fourth check tests the sabotage hypothesis with regressions of delta on inverted stance by thread position, controlling for reply length and topic. The decisive negative outcome would be stable end-to-end models combined with no difference between inverted and literal threads. That would show sarcasm is not a meaningful error source in persuasion prediction, and we would report it as such.