When Explanations Flip Labels: Tracing Run-to-Run Instability in Explain-then-Classify LLMs

Teams increasingly use large language models as classifiers: data annotators, content guardrails, fact-checkers. These models are often asked to write a short justification before giving the label. Two literatures disagree about what that justification does to reliability. Work on explanation-augmented classification reports that written rationales make labels more accurate and less sensitive to how the prompt is worded. Work on output variability reports that hosted models are not reproducible even at temperature zero, and that longer generations give tiny numerical differences more chances to grow into different conclusions. Nobody has measured the question that matters to a practitioner. When the same input is run twice through an explain-then-classify pipeline, does the rationale make the label more likely or less likely to change, and why?

This note proposes a controlled study that answers that question and turns the answer into a tool. The approach runs open models on local hardware, where outputs are exactly reproducible, and then injects small numerical perturbations that do not change the meaning of the computation, such as different batch sizes or floating-point formats. This reproduces the kind of variability seen in cloud serving, but with full control. We can then force two runs to share the same rationale up to any chosen word and record where the label begins to flip. The central hypothesis is that rationales amplify label flips on closed-world deductive tasks, where the model already "knows" the answer from the input, and stabilize labels on tasks that need outside knowledge. If that holds, part of the reported harm of explanations to large-model classification is single-sample noise rather than a real loss of usefulness.

Core Claim

Label instability in explain-then-classify pipelines can be split into two parts. The first is rationale-induced flips: different runs write different arguments, and those arguments lead to different labels. The second is read-out flips: the same rationale is followed by a different label. We claim the first part dominates, and that its size depends on the task type in a predictable way. On e-SNLI, a closed-world inference task, we expect explaining to raise the flip rate relative to direct labeling. On HealthFC and CommonsenseQA, which need knowledge beyond the input, we expect it to lower the flip rate.

A secondary claim concerns detecting unstable inputs. How much two or three sampled rationales disagree in meaning should predict which inputs will flip, and which will be wrong. It should do so at least as well as disagreement among the sampled labels themselves, because rationales can diverge even when the labels happen to agree.

The test for the main claim is a per-position "flip hazard." Fix a reference rationale $r$ for input $x$. Take its first $t$ tokens as $r_{<t}$, let the model continue under a perturbed numerical configuration, and compare the resulting label $\hat y$ with the reference label $y_r$:

$$h_x(t) = \Pr\big[\hat y \neq y_r \mid x,\ r_{<t}\big].$$

Rationale-induced flips correspond to $h_x(0)$ being much larger than $h_x(|r|)$, where $|r|$ is the rationale length. The second quantity is the read-out flip rate when the entire rationale is fixed. The shape of $h_x(t)$ shows whether label-relevant divergences happen early, in how the problem is framed, or late, near the conclusion.

Relation to Prior Work

Three papers motivate the stabilization side of the tension. AnnoLLM injects self-generated rationales into few-shot prompts and finds they reduce sensitivity to prompt wording. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs repeats each medical question five times and finds that deliberative "reflection of thoughts" prompting gives the highest consistency for GPT-4 (62.9%). Both measure variation across inputs or prompts, or across a small number of repeated runs, without isolating where the variation comes from.

Two papers motivate the amplification side, and they constrain the design. Non-Determinism of "Deterministic" LLM System Settings in Hosted Environments shows that temperature-zero runs on hosted models fluctuate in accuracy by up to 15 points, while local Llama-3-8B is fully reproducible. We reuse that local-versus-hosted contrast as our control. An Empirical Study of the Non-Determinism of ChatGPT in Code Generation finds that chain-of-thought prompts increase variability in code outputs. Neither paper studies classification labels conditioned on rationales.

The explanation-consistency methods EC-finetuning and CREST define consistency differently from us. EC-finetuning asks whether an explanation agrees with answers on related questions. CREST asks whether a rationale survives follow-up questions, and uses that to filter training data. We study repeated runs on the identical input. The two notions may come apart, since a model can be logically coherent across related inputs and still unstable on re-execution. GRACE penalizes drift between rationales during reinforcement-learning training for embeddings. It never characterizes how rationales vary at test time.

Finally, Can LLM-Generated Textual Explanations Enhance Model Classification Performance? supplies the puzzle we test. Appending generated explanations dropped zero-shot large-model accuracy on e-SNLI by 7–15%, while small fine-tuned models gained substantially. That study used single runs. We ask how much of the drop survives repeated runs and averaging over rationales. The Effect of Sampling Temperature on Problem Solving in Large Language Models shows that answer accuracy stays flat from temperature 0 to 1 while the reasoning text changes. That is the reason to separate how much the text varies from how much the label varies.

Research Program

The first component is a perturbation harness. It runs two or three open models of different families and sizes, including at least Llama-3-8B and a Qwen 2.5 model, under a grid of meaning-preserving numerical configurations: batch size, batch composition, bf16 versus fp16, and the number of GPUs a model is split across. Temperature-zero decoding is held fixed throughout. Temperature sampling is studied separately as a second source of variability. Hosted APIs are queried ten times per input, following the protocol of Non-Determinism of “Deterministic” LLM System Settings in Hosted Environments Non-Determinism of “Deterministic” LLM System Settings in Hosted Environments, to check that the controlled perturbations produce flip rates in the same range as real serving.

The second component is the branching analysis. For each input we choose a reference rationale and teacher-force its prefixes at a grid of positions. We regenerate under perturbed configurations and estimate $h_x(t)$. We also re-read the label from the complete reference rationale to estimate the read-out flip rate. Results are aggregated by task type, rationale length, and model size.

The third component tests whether the utility gap is real. On e-SNLI and HealthFC we compute accuracy averaged over runs, rather than from one draw, for three pipelines: direct labeling, explain-then-label, and rationale-marginalized labeling. The last one samples $K$ rationales and averages the label probabilities a fixed reader assigns given each rationale. The reader is either the generating model scored deterministically on label tokens, or a small fine-tuned encoder, which (Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study; Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study) found benefits from rationales.

The fourth component is an instability predictor. For a small number of samples per input, we measure disagreement between rationales with an off-the-shelf sentence-embedding model and with entailment checks between rationales. We test whether this disagreement predicts flips and errors on held-out runs.

Baselines

The naive reference point is direct labeling at temperature zero with no rationale, the input-output prompting of Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. That paper reports Fleiss' kappa of 0.982 and 0.984 for GPT-3.5 under this prompting, so it is a demanding stability target. It runs under exactly the same perturbation grid and inputs as every other pipeline. A rationale pipeline "beats" it on stability only if its flip rate is lower with overlapping bootstrap intervals excluded. It beats it on accuracy only if run-averaged accuracy is higher.

The second baseline is single-sample explain-then-label, the setting of Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study. The reported number is a 7–15% zero-shot accuracy drop on e-SNLI for GPT-4o-mini and Qwen 2.5 when explanations are appended. The file gives no absolute e-SNLI accuracy for this setting, and for the flip rate there is no reported result found on Lacuna for repeated-run label flips in explain-then-classify pipelines on e-SNLI, HealthFC, or CommonsenseQA. We therefore re-measure this baseline ourselves rather than quote a figure.

The strongest competitor is label-level self-consistency: sample $K$ rationale-plus-label outputs and take the majority label. We also include its confidence-weighted adaptive variant, Reliability-Aware Adaptive Self-Consistency, which weights samples by low-percentile token confidence and reports up to 70% cost reduction over standard self-consistency on GSM8K with Gemma-3-4B-it while keeping accuracy. These are the obvious way to buy stability with more samples. They are compared to rationale-marginalized labeling at matched total generated tokens, with the same models, perturbation grid, and prompt-tuning budget, which is a fixed small set of templates chosen on a development split. Rationale-marginalized labeling beats them only if it reaches a lower flip rate or higher accuracy at equal token cost.

For the predictor, the competitors are vote entropy over sampled labels and the bottom-10% token-confidence score from ReASC. Both are computed from the same samples.

Three ablations isolate rationale content. First, the label is read after a rationale written for a different input, which tests whether content or mere length drives flips. Second, rationales are truncated to a fixed budget, which tests length against framing. Third, the full rationale is fixed and only the read-out is re-run, which isolates the read-out term.

Why not just…?

Why not just set temperature to zero? That is already the setting under study. (An Empirical Study of the Non-Determinism of ChatGPT in Code Generation; An Empirical Study of the Non-Determinism of ChatGPT in Code Generation) finds 43.64% of CodeContests problems still give differing outputs at temperature zero. (Non-Determinism of “Deterministic” LLM System Settings in Hosted Environments; Non-Determinism of “Deterministic” LLM System Settings in Hosted Environments) finds raw-text agreement near 0% and 15-point accuracy swings for hosted models. Our hosted-API arm checks whether classification labels behave the same way.

Why not just run models locally, where outputs are reproducible? This is the most serious objection. Bitwise reproducibility removes visible flips but not fragility. If a label changes when the batch size changes, a deterministic pipeline is simply reporting one arbitrary draw from a fragile distribution, and an accuracy number from that draw is unreliable. The flip hazard measures exactly this fragility. If our perturbation grid produces negligible flips locally, we will report that local deployment largely solves the problem for these tasks. That would be a useful negative result.

Why not just skip the rationale and label directly? For e-SNLI this may well be the right answer, and our hypothesis predicts it. The evidence is the input-output kappa of 0.98 in Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs and the accuracy drop in Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study. On knowledge-requiring tasks, however, rationales carry accuracy gains. Examples are CAGE's improvement from 54.8% to 64.7% on CommonsenseQA in Explain Yourself! and DynaGuard's 73.1% F1 against 13.1% for LlamaGuard3 in DynaGuard. The study maps where each choice wins rather than assuming one does.

Why not just take a majority vote over sampled labels? This could plausibly match rationale-marginalized labeling on stability. If it does at matched cost, we will say so, and the method contribution shrinks to the diagnosis. The decomposition and flip-hazard findings do not depend on beating voting. Voting also cannot tell agreement reached through consistent arguments from coincidental agreement, which is the "lucky rationale" failure documented by CREST. The predictor experiment tests whether that difference matters for detecting errors.

Proposed Experiments

The benchmarks are e-SNLI and HealthFC, where the utility-gap contrast was reported; CommonsenseQA and BoolQ as knowledge-requiring tasks; and three-class sentiment on SST as a deliberately easy control, given the neutral-class drift observed in Stability Analysis of ChatGPT-based Sentiment Analysis. Metrics are:

  • the label flip rate across runs, meaning the fraction of inputs whose label is not unanimous;
  • Fleiss' kappa;
  • mean and spread of accuracy across runs;
  • the flip hazard curve $h(t)$;
  • for the predictor, the area under the ROC curve for predicting flips and errors.

Three checks decide the outcome.

  1. Crossover by task type. Explain-then-label should flip more than direct labeling on e-SNLI and less on HealthFC and CommonsenseQA. If the ordering is the same on every task, the task-type hypothesis is wrong.
  2. Whether the e-SNLI drop is noise. If the explanation-induced accuracy drop shrinks substantially under run-averaging and rationale marginalization, much of it was single-sample noise. If it persists, it is a genuine cost of explaining.
  3. Whether rationale disagreement adds signal. With two or three samples, rationale disagreement must beat both vote entropy and the token-confidence score at predicting errors. If it does not, label-level signals are sufficient and rationale analysis adds nothing for this purpose.