Do Chain-of-Thought Explanations Actually Reverse Polarity?

Language models are known to get one specific kind of logic wrong. Swapping a word for a more specific one ("laptop" → "MacBook") usually breaks an entailment. Under words like no, not, few or at most, the direction flips: "No student carries a laptop" does entail "No student carries a MacBook." Models trained on ordinary inference data learn the common upward direction and ignore the flip. Separately, models that "think step by step" often write explanations that do not actually produce their answers. They settle on an answer early and justify it afterwards, which is called post-hoc rationalization. This note proposes using the first failure as a precise instrument for studying the second.

The reason is simple. In monotonicity inference, the correct label is a known function of two facts: whether the context is upward or downward, and whether the substituted word is more general or more specific. A good explanation names both facts. We can therefore edit exactly one stated fact inside a model's own explanation, for example rewrite "no is downward-entailing" as "no is upward-entailing." Logic then tells us exactly what the answer should become if the explanation is really driving the decision. That turns faithfulness from a fuzzy notion into a prediction we can check item by item. We then test whether a training signal built from these grounded counterfactuals makes explanations causally binding, compared against simply training on more downward-entailing data.

Core Claim

Our hypothesis is that the upward bias of language models survives inside chain-of-thought as a decorative polarity step. Models will often write a correct sentence identifying the downward operator, yet the final label will not depend on that sentence. The failure should be asymmetric: the stated polarity step should be causally weaker on downward items than on upward ones.

If this holds, two things follow. First, downward-monotone inference is a concrete, cheap and exactly scorable case of the "common cause" pattern, in which the prompt drives the reasoning and the answer separately rather than through each other. Second, the obvious fix of fine-tuning on downward-entailing data may raise accuracy without making the explanation the thing that carries the decision.

What is new is the measurement. Existing faithfulness tests replace or truncate steps and ask whether the answer moves at all. Here, every intervention comes with a logically determined target answer. So we can tell three cases apart: a model that follows its trace, a model that ignores its trace, and a model that notices and explicitly corrects a planted error.

Where This Sits

Three lines of work motivate the gap.

Two pieces of work constrain the design.

  • Formal translation. Faithful Logical Reasoning via Symbolic Chain-of-Thought makes traces faithful by carrying first-order logic alongside natural language, on deduction benchmarks. Do LLMs Game Formalization? shows that translation into a formal proof language such as Lean can be gamed, for instance through fabricated axioms. We therefore keep the trace in natural logic, meaning reasoning directly over surface sentences with ↑/↓ marks, and do not translate into first-order logic.
  • Mitigation by training. FRIT uses direct preference optimization (DPO, a method that trains a model to prefer one of two outputs) over traces whose steps were found causally important by distractor replacement. It reports +7.6 points on GSM8K for Mistral-7B. Our training signal differs in that the counterfactual answer is dictated by logic rather than inferred from whether a distractor happened to flip the prediction.

Research Program

Traces. We elicit traces in a fixed natural-logic format. Each trace states, in order:

  1. the substituted words,
  2. their inclusion relation (more general or more specific),
  3. the monotonicity of the context they sit in,
  4. the composed conclusion.

The format is lightweight and stays in English. It exists so that step 2 and step 3 can be located and edited reliably. We also collect free-form traces, so that the format itself is not the only regime studied.

The central measurement. For each item we generate a trace and then make controlled edits.

  • Polarity edit: flip the stated monotonicity in step 3.
  • Inclusion edit: flip the stated relation in step 2.
  • Control edit: paraphrase the premise restatement, which should change nothing.

After each edit we truncate the trace and force an answer. Let $g(m, r)$ be the natural-logic label for monotonicity $m$ and inclusion relation $r$. The polarity faithfulness of a model is

$$\Phi_M = \Pr\big[\hat y(\text{trace with } m \to \tilde m) = g(\tilde m, r)\big] - \Pr\big[\hat y(\text{control edit}) \neq \hat y(\text{original})\big],$$

where $\hat y$ is the forced answer and $\tilde m$ is the flipped monotonicity. The first term asks whether the answer follows the edited polarity. The second subtracts general sensitivity to any edit. $\Phi_R$ is defined the same way for inclusion edits. We report both quantities separately for upward and downward items. Our hypothesis predicts $\Phi_M^{\downarrow} \ll \Phi_M^{\uparrow}$, while $\Phi_R$ stays comparatively high.

Self-correction confound. A model may ignore an edit because it re-reads the premise and fixes the error, not because it rationalizes. We separate these cases. We let a subset of edited traces continue freely and classify them into two groups: explicit corrections, where the trace says the stated polarity is wrong, and silent overrides. Only silent overrides count as evidence of rationalization.

Complementary probes. We run the early-answering test of Measuring Faithfulness in Chain-of-Thought Reasoning, scored by area over the curve as in On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models. We also train a linear probe that predicts the final label from hidden states at the first trace token. A model that has already committed to an answer before writing will show high probe accuracy there.

Training intervention: trace-consistency preference training. We use HELP premises and build preference pairs from the model's own samples.

  • Preferred: a trace whose stated monotonicity and inclusion are correct and whose label equals $g$ of those stated values.
  • Dispreferred, type 1: a trace that states the correct values but gives a label inconsistent with them, i.e. a decorative step.
  • Dispreferred, type 2: a trace that reaches the right label through a wrong polarity statement, i.e. right for the wrong reason.

We add a second, smaller set of pairs on edited traces. There, the preferred continuation either follows the edited values or explicitly corrects them, and the dispreferred continuation silently overrides them. The aim is to make the stated polarity step load-bearing, not merely to make the label correct.

Baselines

Naive reference. The off-the-shelf instruction-tuned model answers directly and with free-form chain-of-thought, with no training. This gives the accuracy and $\Phi$ values that every method must move.

Strongest accuracy competitor: HELP fine-tuning. This is supervised fine-tuning on HELP with label-plus-trace targets in our format. The traces are produced by a template from HELP's own annotations, following the recipe of HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning.

  • Why it is the strongest: the literature shows HELP is what moves the needle on downward items. It raised BERT's GLUE downward-monotonicity score from −67.5 to 29.8 and RoBERTa's context causal effect to 0.828.
  • Fair comparison: it uses the same HELP examples, the same base model and the same number of gradient steps as our method, with a matched learning-rate sweep.
  • What beating it means: matching its downward accuracy on MED while achieving clearly higher downward $\Phi_M$ and a lower silent-override rate.

Strongest faithfulness competitor: FRIT. We run FRIT as published, with distractor replacement used to label step importance and DPO on faithful versus unfaithful traces. It is applied to the same HELP premises with the same number of preference pairs and the same DPO hyperparameter budget.

  • Why it is the strongest: it is the existing method that targets causal importance of steps and reported accuracy gains rather than losses.
  • Reference numbers: +7.6 points on GSM8K (Mistral-7B) and +6.0 on StrategyQA (Qwen3-8B). There is no reported result found on Lacuna for FRIT, or any chain-of-thought faithfulness metric, on MED, HELP or NLI-XY.
  • What beating it means: higher $\Phi_M^{\downarrow}$ at equal or better accuracy.

Symbolic reference. NeuralLog reports 93.4% on MED. It serves as an accuracy ceiling and is not a direct competitor, because it produces no free-text explanation whose faithfulness could be tested.

Ablations. Three ablations isolate the key ingredient:

  1. Our preference training with logic-dictated pairs replaced by pairs labeled only by answer correctness. This tests whether grounding matters.
  2. Our method without the edited-trace pairs.
  3. Prompting with the natural-logic trace format but no training.

Why not just…?

Why not just fine-tune on HELP? This is the most serious threat, and it is part of the scientific question. Fine-tuning on HELP clearly improves encoder sensitivity to context polarity (Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models). However, post-training is also reported to weaken the link between reasoning and answers (How Likely Do LLMs with CoT Mimic Human Reasoning?). If HELP-trained generative models already show high downward $\Phi_M$ and few silent overrides, our training component adds nothing. We will report that outcome, and the measurement contribution still stands. The head-to-head comparison at matched data and compute decides it.

Why not just use FRIT? FRIT only learns which steps matter when a style-matched distractor happens to change the prediction. Its fact banks are Wikidata and arithmetic facts, not polarity statements. A model that ignores its polarity step will also ignore a distractor placed there, so FRIT would mark that step as unimportant rather than penalize the inconsistency. Our first ablation, answer-correctness pairs versus logic-dictated pairs, tests directly whether this grounding is what helps.

Why not just ask for explicit polarity tags, SymbCoT-style, without training? For accuracy this may well work, and the third ablation measures it. But a stated tag is exactly the kind of step we suspect is decorative. Unveiling Confirmation Bias in Chain-of-Thought Reasoning shows that models override their own valid steps when a prior dominates. Also, Measuring and Mitigating Post-Hoc Rationalization in Reverse Chain-of-Thought Generation found that instructions telling a model to suppress an anchor backfire. If prompting alone yields high $\Phi_M^{\downarrow}$, the training intervention is unnecessary, and we would say so.

Why not hand the problem to a solver or NeuralLog? That improves accuracy but removes the object of study: whether a model's own explanation governs its answer. Formal translation also brings the gaming failures documented by Do LLMs Game Formalization?. Most real use of these models runs through free-text reasoning, so its faithfulness matters regardless of whether solvers exist.

Why not just use a bigger model? Scaling did not fix monotonicity (Monotonicity Reasoning in the Age of Neural Foundation Models), and faithfulness was reported to fall with size (Measuring Faithfulness in Chain-of-Thought Reasoning). We nonetheless measure $\Phi$ across two model sizes within one family, so the scaling prediction is tested rather than assumed.

Proposed Experiments

Models. We use the open-weight models FRIT used, Mistral-7B and Qwen3-8B, plus a larger member of the Qwen family for the size comparison.

Data and splits. - Training: HELP. We hold out operators by training on no/not and testing on few, at most and neither, to check that gains are not memorized operator lists. - Diagnostic evaluation: MED, split into upward and downward items, and CURRICULUM. - Factorial edits: NLI-XY, whose separate monotonicity and inclusion labels make the edits exact. - Transfer: the Japanese numeral inference dataset of Constructing an Annotated Corpus and Inference Dataset for Japanese Numeral Expressions. It tests whether a model that learned to make the polarity step binding for nouns also does so for "not having 3 entails not having 4." - Spillover: AQUA and LogiQA, with the early-answering area-over-curve metric, to check whether faithfulness gains carry over and whether accuracy drops of the kind reported for TruthfulQA by On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models appear.

Metrics. Accuracy split by direction; $\Phi_M$ and $\Phi_R$ split by direction; silent-override and explicit-correction rates; early-answering area over the curve; and first-token probe accuracy.

Decisive checks. 1. The diagnosis. In the untrained models, is $\Phi_M^{\downarrow}$ substantially below $\Phi_M^{\uparrow}$, with $\Phi_R$ comparatively intact? If not, the decorative-polarity hypothesis is wrong, and that negative result is itself informative. 2. The training claim. Does our method beat HELP fine-tuning on downward $\Phi_M$ at matched MED accuracy, and does the first ablation fall back toward HELP fine-tuning? That would show the logic-dictated pairs carry the effect. 3. Generalization. Do gains survive on held-out operators and on numerals? If they do not, the method has taught a lexical trigger rather than a step the answer depends on.