Where Bias Enters a Chain of Thought: Counterfactual Change-Point Segmentation and Local Repair

When a language model reasons step by step toward a biased answer, the bias usually does not appear in the first sentence. At some point in the reasoning, the model's likely answer starts to depend on who the question is about. We propose to find that point directly. After each reasoning step, we ask the model for its answer twice: once with the original prompt and once with the demographic group swapped, while keeping the same reasoning text. This gives a curve of how much the answer depends on demographics as the reasoning unfolds. We then split that curve into a few stable stretches using dynamic programming, an exact method that finds the best way to cut a sequence into contiguous pieces. The cut where the gap jumps marks where bias enters. To repair the trace, we keep everything before that cut and regenerate only the offending stretch, instead of throwing away whole reasoning paths.

This matters because current generation-time fixes work at one of two extremes. COFT filters single tokens against a demographically masked view, and its guarantees hold only per step. BiasTrace samples many full traces and uses a language-model judge to discard biased ones. Neither method locates the step at which reasoning turns demographic-dependent. Neither can tell a trace that drifts into bias from a trace whose answer was biased before any reasoning began. That second case is the post-hoc rationalization documented by Unveiling Confirmation Bias in Chain-of-Thought Reasoning.

Core Claim

A reasoning trace $r = (r_1,\dots,r_T)$ of $T$ sentence-level steps answers a prompt $x$ that names a demographic group. Let $x'$ be the same prompt with the group swapped, as BBQ's templates allow. We define the counterfactual answer gap after step $t$ as

$$g_t = \mathrm{JS}\big(p(a \mid x, r_{1:t}) \,\|\, p(a \mid x', r_{1:t})\big),$$

where $p(a \mid \cdot)$ is the model's distribution over answer options when forced to answer after the prefix, and JS is Jensen–Shannon divergence. The raw curve $g_0,\dots,g_T$ is noisy from step to step. We therefore fit a piecewise-constant model by exact dynamic programming. With $E(i,k)$ the minimum cost of splitting $g_{0:i}$ into $k$ segments,

$$E(i,k) = \min_{j<i}\big\{E(j,k-1) + \textstyle\sum_{t=j+1}^{i}(g_t-\bar g_{j+1:i})^2\big\} + \beta,$$

where $\bar g_{j+1:i}$ is the segment mean and $\beta$ is a per-boundary penalty in the spirit of description length. The recurrence is the standard one from Computational Complexity of Segmentation and the MDL haplotype-block method. Here $T$ is a few dozen sentences, so the quadratic cost is negligible.

The entry boundary is the first cut after which the segment mean rises by more than a margin $\delta$. Before the gaps are compared, their thresholds are calibrated to BBQ's base rates, following Directional Bias Amplification. That paper showed uncalibrated thresholds can pass for bias amplification: calibration cut measured amplification from 0.040 to 0.001.

The claim has three testable parts. First, entry boundaries are causally meaningful: regenerating from them lowers the gap more than regenerating from other points. Second, local repair at the entry boundary matches or beats full-path rejection at a fraction of the tokens. Third, when $g_0$ is already high, meaning the model was biased before reasoning, no local repair helps. This is a falsifiable prediction that makes the confirmation-bias account of the failure mode measurable at the level of individual traces.

Research Program

The work has two parts: a diagnostic and a repair procedure.

The diagnostic runs the probe after every sentence of greedy and sampled chains of thought on BBQ, using Llama3-8B and Mistral-7B. Llama3-8B makes the results comparable with BiasTrace; Mistral-7B is the model on which the confirmation-bias paper found reasoning hurts. Probing adds one short forced-answer pass per step per view, which is cheap with cached prefixes. We then relate entry boundaries to BiasTrace's behavior categories on shared traces. The question is whether entries coincide with "Group Assumption" or "Overthinking" steps. That would indicate whether the probe recovers what the judge labels, without needing a judge.

Repair truncates the trace at the entry boundary. It then samples $m$ continuations of the next segment and keeps the one whose appended gap increment is smallest, before continuing generation normally. The whole loop runs at most twice per trace.

Our segmentation borrows the view of LUA and TXTSEG: cutting a sequence globally with dynamic programming gives cleaner boundaries than local decisions. Those works segment sentences and documents for syntax or classification and never touch social bias. We instead segment a signal built from counterfactual probes over reasoning steps.

The method has a limit we state up front. It still needs a swappable demographic mention, so it shares COFT's dependence on explicitly identified groups. We do not claim to handle unannotated proxies.

Baselines

The naive reference point is plain chain-of-thought decoding. BiasTrace reports that Llama3-8B on BBQ has a bias rate of 13.36% and accuracy of 66.27%. We rerun this with our prompts so that all comparisons share one setup.

The strongest existing method on this exact model and benchmark is Maj-BIASTRACE, which samples several traces, discards those its judge flags, and takes a majority vote. BiasTrace reports it brings the bias rate to 7.36% and accuracy to 77.27% on Llama3-8B. We run it with the same number of samples and report total generated tokens and judge calls alongside the bias numbers. Beating it means matching its bias rate and accuracy at no more than half its token budget, or lowering bias at equal budget.

COFT is the strongest token-level competitor. It reports a 30–55% (median 38%) bias reduction across a suite that includes BBQ, but the file lists no Llama3-8B BBQ-specific figure, so we run it ourselves on identical prompts. We also run a compute-matched best-of-$K$ baseline that scores full traces by their final gap $g_T$. This isolates whether localization adds anything over simply selecting whole traces by the same signal.

Three ablations isolate the key ingredients. The first repairs from a uniformly random step. The second repairs from the first step where raw $g_t$ crosses $\delta$, with no dynamic programming. The third restarts the whole trace with the same extra-token budget.

Why not just…?

Why not just tell the model to avoid stereotypes? Task-complexity probing found such guardrails collapse on complex prompts. BiasTrace found models write "I should not assume…" and then choose the biased answer anyway. We include an instruction-prompted variant of every method so its effect is measured, not assumed.

Why not just sample many traces and filter, as Maj-BIASTRACE does? This is the real threat, and it already works. Our case rests entirely on efficiency and diagnosis. If compute-matched best-of-$K$ scored by $g_T$ equals local repair, then localization buys nothing for mitigation and the contribution shrinks to the diagnostic. We will report that outcome as such.

Why not just threshold the per-step gap instead of using dynamic programming? Crossing a threshold on a noisy curve fires early on spikes and late on slow drifts. Still, a smoothed threshold may do almost as well. The first-crossing ablation settles this directly by comparing repair success and how stable the boundaries are across resampled probes. If the two tie, we will say the segmentation is unnecessary.

Why not just fine-tune for consistency, as in BCT? BCT reports reducing biased reasoning by 86% on held-out tasks for the bias it trained on, and 37% on average across eight held-out biases. It needs weight access and paired unbiased targets. Our method runs on frozen models, and its diagnostic output complements training rather than replacing it. We treat BCT as out-of-regime and do not claim to beat it.

Proposed Experiments

The decisive causal check compares gap reduction after regenerating from the entry boundary against regenerating from random, earlier, and later boundaries on the same traces. If the entry boundary is not significantly better than the nearby alternatives, the localization claim fails.

The second check splits BBQ ambiguous items by the initial gap $g_0$. We predict that repair succeeds on low-$g_0$ traces and fails on high-$g_0$ ones. Failure on high-$g_0$ traces would support the view that confirmation bias shows up at the level of each trace.

Primary metrics are BBQ bias scores and accuracy in both ambiguous and disambiguated contexts, together with generated tokens per item. We also report agreement between entry boundaries and BiasTrace behavior labels.

To measure cost to general utility, we run GSM8K and StrategyQA with repair enabled. COFT stayed within ±0.2 points on these benchmarks, which is the tolerance we adopt. As a check that the probe transfers beyond question answering, we run CrowS-Pairs-style paired prompts that are converted into forced-choice form.