Reasoning Graphons: Measuring and Dialing Post-Hoc Rationalization in Chain-of-Thought

When a language model writes out its reasoning before answering, the written steps often do not cause the answer. The model may settle on the answer from the question alone and then write a plausible justification, a failure called post-hoc rationalization. We propose to describe how a model's steps depend on one another and on the answer using a single smooth object estimated from many traces. For each trace, we record which earlier sentence each sentence actually relies on, which sentences the answer relies on, and how much the answer relies on the question directly. Pooled over many traces, these records form a probability function over relative positions in the trace: the chance that a step two-thirds of the way through depends on a step one-third of the way through. This is a graphon, the continuous limit object that graph-augmentation methods such as Mixup for graphs use to blend whole families of graphs. Here it summarizes a model's reasoning structure on a task. It can be compared across traces of different lengths and across models. Most importantly, it can be interpolated toward a strictly step-by-step target structure and sampled as attention masks. That turns faithfulness from something we only measure into a dial we can turn, while recording what happens to accuracy.

The payoff is a test of the field's central unresolved tension. On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models reports that steering toward faithfulness costs up to 20% accuracy on TruthfulQA. FRIT reports a 7.6-point GSM8K gain from faithfulness training. Our hypothesis is that the sign of that trade-off can be read off the estimated reasoning graphon before any intervention. It should be negative when the answer's dependence on the steps carries little information, and positive when step structure exists but the model underweights it.

Core Claim

Post-hoc rationalization is a structural property of the sentence-level dependency graph of a trace. Because reasoning sentences come in a natural order, that structure admits a directed graphon whose node coordinates are given rather than estimated. This removes the node-alignment problem that forces G-Mixup to sort nodes by degree and forces MomentNet to work in motif-count space.

Write $u \in [0,1]$ for the relative position of a sentence in its trace. The reasoning graphon has three parts. $W(u,v)$ for $v<u$ is the probability that the sentence at $u$ causally depends on the sentence at $v$. $a(u)$ is the probability that the answer depends on the sentence at $u$. $b$ is the probability that the answer depends directly on the question, bypassing the steps. A strictly serial reasoner has $W$ concentrated just below the diagonal, $a$ concentrated near $u=1$, and $b$ near zero. A post-hoc rationalizer has small $a$ and large $b$. We interpolate between a model's estimated structure $(W_M, a_M, b_M)$ and a serial target $(W_S, a_S, b_S)$:

$$W_\lambda = (1-\lambda) W_M + \lambda W_S, \quad a_\lambda = (1-\lambda) a_M + \lambda a_S, \quad b_\lambda = (1-\lambda) b_M + \lambda b_S,$$

where $\lambda \in [0,1]$ is the interpolation weight. This family yields two things. First, a per-trace score: the $\lambda$ that maximizes the likelihood of that trace's observed dependency graph, which locates the trace on the rationalization-to-serial axis. Second, an intervention: sentence-level attention masks sampled from $W_\lambda$, which force the model to reason with a structure a controlled distance from its own.

Two properties set this apart from prior work. Interpolating toward the serial target preserves the model's native step-to-step patterns except along the axis of interest. And unlike a hard bottleneck, $\lambda$ is continuous.

Relation to Prior Work

The motivating evidence comes from three lines of work. How Likely Do LLMs with CoT Mimic Human Reasoning? uses causal mediation to show that the question drives steps and answer in parallel. Can Aha Moments Be Fake? finds many steps have no causal effect. Measuring Faithfulness in Chain-of-Thought Reasoning shows that truncation, injected mistakes and filler tokens often leave answers unchanged, with more post-hoc behavior on ARC-Easy than AQuA. All three report behavior; none gives a representation that can be compared across traces of different lengths or used to intervene.

The method for building graphs comes from Thought Anchors, which extracts sentence-level dependency graphs by suppressing attention to a sentence and measuring downstream logit shifts. We reuse that procedure, but treat the graphs as samples from a population rather than as per-trace visualizations. Verifying Chain-of-Thought Reasoning via Its Computational Graph also reads reasoning off graph structure, flattening transcoder attribution graphs into hand-chosen statistics for a boosted classifier. That work operates at the level of internal features and targets step errors. We operate at the sentence level, target faithfulness, and model the distribution rather than summary scalars.

The graphon machinery comes from three sources. We borrow interpolation from G-Mixup. We borrow smooth neural representations of $W$ from Implicit Graphon Neural Representation and Scalable Implicit Graphon Learning. We borrow the observation, from From Moments to Models, that one dataset can hold several structural populations, which matters because a model may rationalize on some problems and reason serially on others.

Two results constrain the design. A Few Moments Please proves that mixing graphons is not the same as mixing their motif counts. We interpolate graphons because we need to sample masks from the mixture. All of these methods also assume undirected, unordered, dense graphs with motif counts that are noisy on small graphs. We address both: the given ordering handles direction and alignment, and pooling many short traces into one smooth estimate handles the noise from small graphs.

On the intervention side, Markovian Transformers blind the answer to the question entirely. That corresponds to our $b=0$ endpoint, but it is trained from that constraint and has no intermediate setting. FRIT builds preference pairs from step interventions and trains with Direct Preference Optimization, changing weights without describing structure.

Research Program

The first stage builds the data. For Qwen3-8B and Mistral-7B (the models FRIT used) and Llama 3.1 8B (CRV's model), we sample chain-of-thought traces on GSM8K, SVAMP, AQuA, LogiQA, StrategyQA, ARC-Easy and ARC-Challenge, plus CRV's synthetic arithmetic, where step dependencies are known by construction. Each trace is split into sentences. We then apply Thought Anchors-style attention suppression to each sentence and to the question span, producing a directed dependency graph with question and answer nodes. A cheaper approximation from late-layer attention to earlier sentences is computed alongside, so we can report how much fidelity it loses.

The second stage estimates graphons. We fit $W$, $a$ and $b$ per model and task with an implicit neural representation over positions, then fit a mixture of such graphons in the spirit of graphon-mixture-aware Mixup. The number and weight of components measures how many distinct reasoning populations a model has on a task. Each trace receives its maximum-likelihood $\lambda$ under the interpolation family.

The third stage intervenes. At inference time, we sample sentence-level attention masks from $W_\lambda$ for $\lambda$ on a grid. The answer-to-question mask is drawn with probability $b_\lambda$, and step-to-step masks are drawn from $W_\lambda$. We record accuracy and faithfulness as curves over $\lambda$.

The fourth stage trains. We fine-tune with masks sampled at random $\lambda$, a structural analogue of Mixup, and test whether the model internalizes the serial structure once masks are removed.

Baselines

The naive reference for the detection question is a predictor using only trace length and question length. Length is a known correlate of how much reasoning a model does, and any structural score must beat it. For intervention, the naive reference is the unmodified model with ordinary chain-of-thought.

The strongest existing per-trace faithfulness measure is the early-answering area-over-curve score of Tanneru et al., built on Lanham et al.'s truncation test. It is cheap, needs no internals, and is what the field reports. We compute it on exactly the same traces. The ground truth is whether an injected arithmetic or logical mistake, in the style of Lanham et al., changes the final answer. Beating it means higher area under the ROC curve for predicting mistake-sensitivity, especially when training on short traces and testing on long ones, since length transfer is what the graphon representation is meant to provide. No reported result found on Lacuna for early-answering scores as a per-trace predictor of mistake-sensitivity, so we establish both numbers ourselves.

A second detection baseline follows CRV's recipe. We extract hand-chosen statistics (density, path lengths, centrality) from the same dependency graphs and train gradient boosting on them. CRV reports AUROC 92.47 against 76.45 for the best baseline on synthetic arithmetic error detection (Verifying Chain-of-Thought Reasoning via Its Computational Graph). That figure is for a different target, step errors, so it shows the recipe is strong but is not directly comparable.

The key ablation for detection is the bare scalar $b$ per trace, meaning direct question-to-answer dependence with no graphon at all. If that scalar matches the full model, the graphon adds nothing to detection.

For intervention, the strongest training-based competitor is FRIT. It reports a 7.6-point GSM8K accuracy gain for Mistral-7B and a 6.0-point StrategyQA gain for Qwen3-8B (FRIT). The architectural competitor is the Markovian Transformer, which reports GSM8K accuracy rising from 19.6% to 57.1% (Markovian Transformers) in its own training setting. We rerun both on the same base models and data, under a matched fine-tuning budget measured in gradient steps and tokens, with the same hyperparameter search size for every method. Beating them means equal or better accuracy with higher mistake-sensitivity.

The ablation that isolates interpolation is uniform bypass knockout. Here we mask only the answer-to-question attention with probability $1-\lambda$ and leave step-to-step attention untouched. It is the simplest form of the same dial.

Why not just…?

Why not just use early answering as the faithfulness score? It may well suffice on fixed-length tasks, and we treat that as a live possibility. Its known weakness is confounding. A flat probability curve can mean the steps are decorative, or that the problem was easy and every step agrees with the answer. Lanham et al.'s contrast between ARC-Easy and AQuA is consistent with either reading. The dependency graph separates the two: decorative steps show missing step-to-answer edges, while redundant but consistent steps show the edges present. The mistake-injection ground truth decides which reading is right. If early answering matches our score there and under length transfer, we will report it as the practical choice and keep the graphon only for the intervention.

Why not just knock out the question-to-answer path at a variable rate? This is the most serious threat to the interpolation claim. A single bypass knob might trace the same accuracy curve as the full graphon family. The argument for $W_\lambda$ is that forcing serial structure also changes which earlier steps each step can see. A model whose native pattern refers back to a plan sentence (the "anchors" Thought Anchors highlight) may break under a uniform chain mask but not under an interpolation that starts from its own $W_M$. We compare the two at matched expected mask density. If the curves are statistically indistinguishable on every task, we drop the step-structure component of the intervention and keep the simpler knob.

Why not just train with FRIT or a Markovian bottleneck? Both change the model without explaining when faithfulness helps or hurts accuracy, which is the open question between Tanneru et al. and FRIT. Our claim is predictive. From $a_M$ and $b_M$ estimated before any intervention, we predict the sign of the accuracy change under forced serial structure for each task. If those predictions fail, the structural account of the trade-off fails, regardless of how the training comparison comes out.

Why not just train a graph neural network on the dependency graphs? A network trained on the graphs would likely detect rationalization within a distribution. But it gives neither a sampler for interventions nor a model-level object that can be compared across models of different sizes. Comparing across sizes is needed to test the inverse-scaling claim that larger models rationalize more. We include it as an additional detection baseline in the length-transfer split.

Proposed Experiments

The first decisive check is detection under length shift. We train every scorer on traces with eight or fewer sentences and test on longer ones, across all seven benchmarks and three models. The target is mistake-sensitivity, and the metric is area under the ROC curve against early answering, CRV-style features, the bare bypass scalar and the length-only predictor. On synthetic arithmetic we additionally check that the estimated $W$ recovers the known dependency structure.

The second check is the dose-response test of the accuracy-faithfulness trade-off. For each model and task, we plot accuracy and mistake-sensitivity over $\lambda$ under graphon-sampled masks and under uniform bypass knockout. We register in advance the prediction that recall-heavy tasks (ARC-Easy, StrategyQA, and TruthfulQA as an external check against Tanneru et al.'s reported drop) lose accuracy as $\lambda$ rises, and that GSM8K, SVAMP and AQuA gain or hold. We also register that the size of the change correlates with the step-to-answer information in the estimated $a_M$.

The third check is structural training. We fine-tune with masks sampled at random $\lambda$, then evaluate with no masks on GSM8K, SVAMP and StrategyQA against FRIT, the Markovian bottleneck and the uniform-knockout ablation. We report accuracy, the early-answering score and mistake-sensitivity, and re-estimate the graphon after training to test whether $b$ actually fell.

The fourth check applies the mixture fit across a small and a larger member of one model family. It tests whether the larger model places more weight on the bypass-dominated component, giving a structural reading of the inverse scaling Lanham et al. observed.

The proposal fails cleanly in three ways. The bare bypass scalar could match the graphon score in detection. Uniform knockout could match graphon interpolation in the dose-response curves. Or the pre-intervention graphon could fail to predict the sign of accuracy change. We will report whichever of these occurs.