Gated Reasoning Vectors for Sarcasm-Flipped Stance in Arabic Tweets

Stance detection asks whether a tweet's author is for, against, or neutral toward a target such as "COVID-19 vaccine." Sarcasm makes this hard because it flips the surface meaning: "great, another mandatory jab, I just love it" uses positive words to express opposition. Large language models handle such cases better when they first write out a short chain of reasoning, called chain-of-thought (CoT) prompting. That reasoning costs many extra generated tokens per tweet. A recent line of work shows that the effect of demonstrations or of a reasoning prompt can be compressed into a single vector added to the model's hidden activations. Theory predicts that one such vector cannot represent an input-dependent flip. We propose to test that prediction where it matters most, on sarcasm-driven stance reversals. We also propose a fix: a two-part steering vector, where the second part is switched on by a probe that reads the model's own internal sarcasm signal.

The testbed is MAWQIF, a set of 4,121 multi-dialectal Arabic tweets labeled at once for stance, sentiment and sarcasm. It is introduced in StanceEval 2024: The First Arabic Stance Detection Shared Task. Because every tweet carries all three labels, we can isolate the "flip" tweets, where literal sentiment and stance disagree, and ask whether compression loses exactly those. The practical goal is stance detection with zero added inference tokens that keeps the gains of reasoning on figurative tweets. The scientific goal is the first test of the rank-one limit of task vectors on a natural, linguistically motivated inversion rather than a synthetic one.

Core Claim

Understanding Task Vectors in In-Context Learning proves that a single task vector acts as a rank-one transformation. On a synthetic uppercase/lowercase bijection, accuracy falls from 100% with full in-context learning to 55% with a single vector. Sarcasm-modulated stance has the same structure: a mapping from surface sentiment to stance whose sign depends on the input.

We claim three things. First, compressed vectors, whether taken from few-shot demonstrations or from CoT, will keep most of the accuracy of explicit prompting on literal tweets but lose disproportionately on MAWQIF's sentiment–stance flip subset. Second, adding a second vector, scaled by the model's own sarcasm probe, recovers most of that loss. Third, this gated vector closes most of the gap to explicit structured CoT at no extra token cost.

The claim is falsifiable at each step. If explicit CoT does not beat direct few-shot prompting on MAWQIF, there is nothing worth compressing. If a single learned vector already handles flips, the rank-one concern does not carry over to natural language steering. Both outcomes would be reported as findings.

Relation to Prior Work

Three groups of prior work bear on this proposal, each in a different role.

The first group motivates it. Several papers show that reasoning about sarcasm helps stance, but all of them do so with explicit text generation or full training, and none on Arabic.

  • Chain of Stance decomposes stance into a six-step prompt that includes an emotional and rhetorical analysis step. It reaches 79.84 F1 few-shot on English SemEval-2016.
  • Sarcasm-R1 trains Qwen2.5-7B with reinforcement learning over linguistic, contextual and emotional-reversal dimensions. It finds emotional reversal to be the main driver. - Chain-of-Thought Embeddings for Stance Detection finds that LLM reasoning is often right while the final label flips. It filters the output through a fine-tuned RoBERTa.

  • Intermediate-Task Transfer Learning and Sarcasm Detection as a Catalyst show that sarcasm pre-training fixes 85% of sarcastic tweets that a stance model had misclassified. They do this with encoder fine-tuning.

We keep their reasoning content but remove generation at inference time.

The second group supplies the method.

  • In-Context Learning Creates Task Vectors extracts a hidden state from demonstrations and patches it into a query-only forward pass, keeping 80–90% of in-context accuracy. It was tested only on single-token toy tasks.
  • CoT Vectors builds two kinds of reasoning vector. One is the mean activation difference between CoT and direct prompts. The other is a learnable vector of about 4K parameters, distilled from a CoT teacher and injected into shallow layers, which beats LoRA on math benchmarks. It also warns that injection into the volatile middle layers degrades output.

We reuse both extraction recipes. We move them to pragmatic, dialectal text, which neither paper studies.

The third group constrains the design. The rank-one result above rules out a single static vector as the final answer. The multi-vector remedy in that paper (TaskV-M) injects several vectors regardless of the input. Ours differs because the second component is gated per input by a learned sarcasm probe.

On the Arabic side:

  • SMASH found that prompting in Arabic caused hallucinations, and that English instructions with Arabic topic strings worked better. It reached a test macro F1 of 0.8041 zero-shot with Command-R, and explicitly left few-shot prompting untested. We therefore write the reasoning scaffold in English.
  • CUFE and MGKM fine-tune Llama-3-8B and GPT-3.5 directly, without reasoning chains.

Research Program

We work with two open-weight models that the file's sources already use on these tasks: Llama-3-8B-Instruct, used by CUFE and MGKM, and Qwen2.5-7B-Instruct, used by Sarcasm-R1. Both are needed because we must read and edit hidden states. All results use MAWQIF's official splits and the shared task's macro F1, broken down by target.

Teacher reasoning chain. We first build a structured, English-language reasoning chain for each training tweet. It has four steps:

  1. A literal English gloss of the dialectal tweet.
  2. The surface sentiment toward the target.
  3. An explicit check for contrast or reversal between the words and the evident intent, adapted from the emotional-reversal step of Sarcasm-R1 and the emotional-analysis step of Chain of Stance.
  4. The stance.

The model generates these chains itself from a handful of hand-written demonstrations. We keep only chains whose final stance matches the gold label. This filtering addresses the label-hallucination problem reported for CoT stance detection.

Diagnostic subsets. Using MAWQIF's sentiment and sarcasm labels, we partition the test set into three groups:

  • literal tweets, where sentiment and stance agree;
  • flip tweets, where positive sentiment comes with Against stance or negative sentiment with Favor;
  • sarcastic tweets, which overlap heavily with the flip group.

We report subset sizes rather than assume them. Because sarcasm is a small minority in MAWQIF, every subset result carries bootstrap confidence intervals.

Vectors. We build three kinds of compressed vector:

  • extracted task vectors from few-shot demonstrations, following the task-vector recipe;
  • extracted CoT vectors, the mean difference between CoT-prompted and direct-prompted activations;
  • learnable CoT vectors, distilled from the teacher chains into a shallow layer following the CoT Vectors recipe.

Gated vector. The proposed method adds a second learnable vector, scaled by a probe:

$$h_\ell(x) \leftarrow h_\ell(x) + v_0 + \sigma\big(w^\top h_{\ell'}(x)\big)\, v_1$$

Here $h_\ell(x)$ is the hidden state of tweet $x$ at injection layer $\ell$, and $v_0$ is the shared reasoning vector. $v_1$ is the "reversal" vector. $w$ is a linear probe applied at an earlier or equal layer $\ell'$, trained on MAWQIF's sarcasm labels with class reweighting. $\sigma$ is the logistic function.

We train $v_0$, $v_1$ and $w$ jointly. The training loss is distillation toward the teacher's stance distribution plus the probe's sarcasm loss. The added parameter count is on the order of three hidden-state widths. The design makes the steering input-dependent with rank two, which is the smallest change that the rank-one argument says should suffice.

Mechanistic check. We measure where stance flips become decodable inside the network. A linear probe for stance is trained at each layer, separately on the flip and literal subsets. If the probe recovers the correct stance on flip tweets even in runs where the output label is wrong, the hidden state contains the answer and the failure lies in how it is read out.

Baselines

The naive reference point is direct zero-shot classification by the same open model, using the SMASH prompt format (English instructions, Arabic target). It costs no extra tokens and sets the floor every method must clear.

The strongest published systems set the ceiling. On MAWQIF, the best reported score is AlexUNLP-BH at 84.38% macro F1, from StanceEval 2024. That system's internals are not described in the file, so we cite it only as a ceiling.

The comparable open-weight, parameter-updating competitor is LoRA on Llama-3-8B. CUFE reports 0.7647 average macro F1, and MGKM reports 74.33% for Llama-3-8B-Instruct. We will rerun LoRA ourselves on the same splits and base model, giving it the same hyperparameter-search budget as our vectors, measured in GPU-hours and number of configurations. We do not compare against the reported numbers directly, since preprocessing differs. LoRA is the strongest competitor on the question we care about: does a tiny activation-space adapter match a weight-space adapter when both see the same supervision?

We also fine-tune an encoder, AraBERT-twitter, multi-task with sarcasm labels. ANLP RG reports 78.08% stance macro F1 and 59.51% sarcasm macro F1 for this setup in ANLP RG at StanceEval2024. It is a cheap, dialect-aware model, and if it wins on the flip subset, the LLM route is unjustified.

Explicit structured CoT is the accuracy target that compression tries to preserve. We run it with the same demonstrations as the teacher, with and without retrieval of similar demonstrations, and record tokens per tweet. We found no reported result on Lacuna for few-shot or CoT prompting on MAWQIF. These runs are therefore new reference numbers, not reproductions.

Three ablations isolate the key ingredients:

  • a single learnable vector $v_0$ with no gate;
  • two vectors with the gate fixed at 0.5 for every input, which matches parameter count but has no input dependence, close in spirit to TaskV-M;
  • a gate driven by the probe's predicted sarcasm label written into the prompt as text, rather than applied as a vector.

The proposal counts as beating the baselines only if three conditions hold. On flip-subset macro F1, the gated vector must beat the single vector by a margin outside the bootstrap confidence interval. It must recover at least most of the gap between direct prompting and explicit CoT. And it must match or exceed our LoRA rerun on overall macro F1 with no added inference tokens.

Why not just…?

Why not just use few-shot CoT prompting and accept the token cost? If cost does not matter, this may be the best accuracy option, and we run it as the target. But MAWQIF has no reported number for it, and the evidence that CoT helps stance and sarcasm comes from English benchmarks. Chain of Stance's 79.84 F1 is on SemEval-2016, and Sarcasm-R1's 72.3 macro F1 is on iSarcasm. Our first experiment establishes whether CoT helps on dialectal Arabic at all. If it does not, we drop the compression program and report that negative result.

Why not just fine-tune with LoRA, or a multi-task Arabic encoder? Both are real threats, and both are rerun under matched budgets. On MAWQIF, Llama-3-8B with LoRA reached 0.7647 (CUFE), which trails zero-shot Command-R at 0.8041 (SMASH). On math benchmarks, CoT Vectors found a 4K-parameter vector gaining 4.6 points against LoRA's 2.9. Neither result is on our task, though. The encoder's 59.51% sarcasm macro F1 against 94.83% accuracy suggests it largely ignores the minority sarcastic class. If either baseline matches the gated vector on the flip subset, the vector's only remaining advantage is parameter count, and we will say so.

Why not just learn one CoT vector? Shallow injection passes through nonlinear layers anyway. This is the most serious objection. The rank-one result was proven for linear attention on synthetic tasks. A vector added early can be transformed differently for different inputs by the layers above it. The ungated single-vector ablation tests this directly. If it matches the gated version on flip tweets, we conclude that the bottleneck does not bind for shallow learned steering in natural language. That would be a useful correction to how the theory is currently applied, and the extracted-vector diagnostic still shows whether the bottleneck binds for non-parametric vectors.

Why not just tell the model the tweet is sarcastic? A textual sarcasm hint is the cheapest form of gating. It is included as an ablation, using the same probe so that detection quality is held equal. If it does as well as vector gating, the useful contribution is the probe, not the steering.

Proposed Experiments

The experiments are ordered as gates; each depends on the outcome of the one before.

Experiment one establishes the reference ladder on MAWQIF for both models: direct zero-shot, few-shot, few-shot with retrieval, and structured CoT. All four are scored on the full test set and on the literal, flip and sarcastic subsets, with token counts recorded. The continuation condition is that CoT's gain over direct prompting is concentrated on the flip subset.

Experiment two is the rank-one diagnostic. For extracted task vectors and extracted CoT vectors, we compute the ratio of vector accuracy to explicit-prompt accuracy separately on the literal and flip subsets, sweeping the injection layer. The prediction is a markedly lower ratio on flip tweets. A uniform drop across subsets would contradict the inversion hypothesis.

Experiment three is the head-to-head comparison: learnable single vector, gated two-vector, fixed-gate two-vector, textual hint, LoRA rerun and multi-task AraBERT. All use matched data and tuning budgets, with five seeds and bootstrap confidence intervals. Results are reported overall, per target, and on the flip subset. Digital transformation gets particular attention, since CUFE's score of 0.6539 on that target shows it is the hardest.

Experiment four probes where the decision forms. Per-layer stance probes on flip and literal tweets, with and without steering, test whether gating moves the point where the correct stance becomes linearly readable. Pairing these probes with the steering results also tests whether failures sit in the volatile middle layers that CoT Vectors warns about.

Experiment five checks transfer. Vectors and probes trained on two MAWQIF targets are applied to the held-out third target. The question is whether the reversal component $v_1$ captures sarcasm as a general operation or only target-specific vocabulary.

The single most decisive result is the flip-subset comparison between the gated and ungated learnable vectors.