Large language models change their answers when a prompt gains a trailing space, a greeting, or a different requested output format. Existing robustness benchmarks count these flips and report how much accuracy varies, but they cannot say what kind of failure a flip is. A model might have a sharp, sensible decision rule whose threshold simply slides when the wording changes. Its rule might blur. Or it might have no coherent rule near the threshold at all. These are different defects that call for different fixes, and a flip count cannot tell them apart. This proposal borrows the main tool of categorical perception research: present stimuli along a controlled continuum and fit an S-shaped "identification curve" to the responses. Categorical perception is the effect, studied in speech and vision, where smooth physical changes are perceived as discrete jumps at a category boundary. Using this tool, every wording change can be described by what it does to the model's decision curve, which reduces brittleness to a few interpretable parameters.
The second ingredient is a task hierarchy. We build tasks in three tiers. The first reads a single stated quantity against a threshold. The second compares two quantities. The third composes two conditions under a rule. Each tier has its own continuum, so we can ask at which tier wording sensitivity enters. We also ask whether the curve at the top tier can be predicted from the curves at the lower tiers. Finally, we read the same continua inside the model's hidden layers. This lets us check the information-theoretic prediction that a well-organized classifier stretches its representation along meaning near the boundary and compresses it along irrelevant variation. If that prediction holds, it gives a mechanistic reading of what brittleness is: the representation is also stretched along form.
Core Claim
Fix a reasoning tier and a decision boundary. Most of the prediction flips caused by meaning-preserving prompt edits come from a shift in where the boundary sits, not from the boundary blurring or breaking apart. These shifts grow at higher tiers of the task hierarchy. Two consequences follow if the claim is right. First, the probability that an item flips should be predictable from its distance to the boundary together with the measured shift. Second, a cheap correction that re-centers the boundary using a few anchor items should restore much of the lost consistency. Few-shot exemplars and chain-of-thought prompting do not target the boundary location, so the correction should beat them. If blur or fragmentation dominates instead, re-centering will fail. That outcome is equally informative, because it means the model lacks the categorical compression that Information theoretic study of the neural geometry induced by category learning shows optimal categorizers must have.
Position Relative to Prior Work
Three robustness studies motivate the work by documenting the problem without decomposing it. Brittlebench applies typos, padding, personas and paraphrases to six benchmarks and splits score variance into item difficulty and prompt sensitivity. POSIX measures how output log-likelihoods shift across 60 variants of each prompt. The Butterfly Effect of Altering Prompts counts flips under 24 edits. All three sample isolated points in prompt space and score each benchmark as one flat task. We sample along structured lines, and we score each tier separately.
Do Prompt-Based Models Really Understand the Meaning of Their Prompts? shapes the design. It found that irrelevant instructions perform about as well as instructive ones, but that reversing the answer words costs a median 22.2% of accuracy. Answer-word swaps therefore belong in our perturbation set as a separate, strong family.
The method comes from the categorical perception literature. Human studies fit sigmoid identification curves along synthesized continua, for example the 13-step vowel continuum in The Influence of Language Experience on the Categorical Perception of Vowels. Categorical Perception: A Groundwork for Deep Learning shows that vision networks become more categorical with depth. Bonnasse-Gahot and Nadal show that the sensitivity of the representation peaks at the class boundary on an MNIST 4-to-9 morph. Both of those papers work only on images.
The Geometry of Categorical and Hierarchical Concepts in Large Language Models finds that hierarchical concepts in Gemma-2B and LLaMA-3-8B occupy orthogonal subspaces. It reaches this from clean WordNet words at the final layer. We reuse its approach of extracting concept directions with linear discriminant analysis, but we apply it at intermediate layers and under perturbation.
On the hierarchy side, FACT and ToolBeHonest show steep performance drops between tiers. Both use unperturbed inputs. LogiQA reports that fine-tuned RoBERTa reaches 55% on categorical reasoning but 17.11%, below chance, on sufficient-conditional reasoning. That gap is a natural external test of whether composition-tier fragility transfers to natural data.
Research Program
We generate templated items whose correct answer is set by a numeric quantity $x$ stepped across 13 values that span a threshold. The answer flips at a known point.
At tier 1, the model reads one stated value and judges it against a threshold, for example: "The reading is 37. Does it exceed 40?" At tier 2, the model compares two stated quantities, and the continuum runs over their difference. At tier 3, the model composes two conditions under a rule, for example: "If the reading exceeds 40 and the pressure is below the limit, the valve opens. Does it open?" One condition is held satisfied, and $x$ moves the other across its threshold. This tier also includes a sufficient-conditional variant that mirrors LogiQA. Surface entities, units, and thresholds vary across many template instances, so the curves are not tied to one string.
Every form perturbation is applied identically to every step of a continuum. The perturbations are drawn from the Brittlebench and Butterfly families: whitespace, greetings, personas, requested output formats, LLM paraphrases (filtered by back-checking that the answer logic is unchanged), and answer-word swaps. We add digit-format changes such as "37", "thirty-seven", and "3.7e1". These change form but not content, a weakness Akhtar documents for numeric data. A second, graded form axis stacks 0 to k edits, which lets us ask whether boundary shift grows smoothly with the amount of surface change.
For each model, tier, and perturbation $p$, we fit the probability of answering "yes" as a function of $x$:
$$P(\text{yes}\mid x,p)=\lambda_p+(1-2\lambda_p)\,\sigma\!\left(\frac{x-\mu_p}{s_p}\right),$$
Here $\sigma$ is the logistic function, $\mu_p$ is the boundary location, $s_p$ is the boundary width, and $\lambda_p$ is the lapse rate, meaning the rate of errors far from the boundary. Fragmentation is measured as the number of non-monotonic reversals and the residual variance left unexplained by the sigmoid. The effect of a perturbation then splits into three parts: translation ($\mu_p-\mu_0$), blur ($s_p/s_0$), and fragmentation.
The prediction to test is that flips concentrate where the clean curve and the perturbed curve disagree. Concretely, an item's flip probability should track $|P(\text{yes}\mid x,0)-P(\text{yes}\mid x,p)|$, computed from the fitted parameters.
Inside the model, we run open-weight base and instruct pairs: Llama 3.1, Qwen 3, Gemma-2B, and LLaMA-3-8B. At each layer we compute two distances along the class direction found by linear discriminant analysis. The first is the distance between hidden states at adjacent continuum steps, which is the model's analogue of a discrimination curve. The second is the displacement caused by each form edit at a fixed step. A "misplaced stretching" index is the ratio of form displacement to the adjacent-step content distance at the boundary. Categorical perception predicts that this ratio is small and shrinks with depth. Brittleness predicts that it is large somewhere, and the layer where it becomes large shows where form leaks into the decision.
Including base and instruct pairs bears directly on a conflict in the literature. POSIX finds instruction tuning increases sensitivity in open-ended generation, while Webson and Pavlick find instruction-tuned models insensitive to instruction semantics. The decomposition can reconcile these if, for example, instruction tuning narrows boundary width while making the boundary location more dependent on wording.
To test how tiers couple, we predict each tier-3 curve from the tier-1 and tier-2 curves of its two component conditions, assuming the conditions combine independently. The excess of observed tier-3 blur or shift over this prediction measures error added by composition itself. This quantifies the tier-coupling problem that hierarchical benchmarks leave open.
Baselines
The naive reference point is the raw flip rate and accuracy change per perturbation, scored over the same items with tiers pooled. This is the quantity the Butterfly study reports. It found over 500 flips in gpt-3.5-turbo-1106 from a single trailing space, and at least 10% of predictions changed by output-format requests.
The strongest existing analysis is the Brittlebench variance decomposition, which reports prompt sensitivity accounting for up to 50% of performance variance in open-weight models and accuracy drops of up to 12%. We will run its decomposition on exactly our items, perturbations, and models. The proposal beats it if tier-wise boundary parameters predict held-out item flips with higher AUROC than item-difficulty and prompt-sensitivity terms fitted on the same training split.
The strongest simple competitor is each item's clean confidence margin, the log-odds gap between its two answers, used as a flip predictor. We give it the same held-out split and no extra tuning. Our model must beat margin alone, and must also beat margin combined with perturbation identity, before boundary location adds anything.
POSIX supplies a log-likelihood sensitivity score computed over the same variants. Its reported anchoring result, template sensitivity on MMLU for Llama-2-7b falling from 1.12 at 0-shot to 0.513 at 1-shot, is the benchmark for the intervention comparison below.
Two ablations isolate the key ingredients. The first keeps our perturbations but samples quantities at random instead of along an ordered continuum, which removes the curve fit and leaves the same data budget. The second uses only tier-3 items, which removes the hierarchy. No reported result was found on Lacuna for flip prediction or psychometric boundary fitting on threshold continua of this kind, so all baselines will be run fresh under matched decoding at temperature 0, with identical item counts.
Why not just…?
Why not just use Brittlebench-style variance decomposition? It answers how much variance comes from prompts. It does not answer whether that variance comes from a sliding threshold, a blurred one, or a broken one, and it does not say at which tier the variance enters. Those distinctions matter because they predict which fix works. The random-sampling ablation tests directly whether the ordered continuum adds predictive information beyond the same perturbations scored flatly.
Why not just use the confidence margin, since low-margin items are the ones that flip? This is the most serious threat. It may explain much of the flip rate, because items near the boundary have small margins by construction. The difference lies in what each predicts about direction. A margin predicts that a flip is likely but not which way it goes. A boundary shift predicts the direction of every flip from the sign of $\mu_p-\mu_0$, and it predicts flips at moderate-margin items when the shift is large. If boundary parameters add nothing over margin plus perturbation identity on held-out items, we will report that brittleness on these tasks is essentially a margin effect, and treat the tier and layer analyses as the remaining contribution.
Why not just add a few-shot exemplar or chain of thought? POSIX finds that one exemplar roughly halves template sensitivity. Brittlebench finds that few-shot prompting increases formatting brittleness and that chain of thought reduces accuracy drops by only about 17%. We run both as interventions on the continua and record whether they act on boundary location, boundary width, or lapse rate. The decisive comparison against them is the re-centering test described next.
Decisive Checks
The first check is re-centering. For each perturbation we estimate a single logit offset from three anchor items near the boundary, then apply it to the rest of that perturbation's items. If translation is the dominant failure, this correction should recover more of the lost agreement than 1-shot anchoring or chain of thought at equal or lower inference cost. If blur or fragmentation dominates, it should fail. The prediction is falsifiable in both directions.
The second check is transfer. The synthetic results predict a larger boundary shift at the composition tier than at the threshold tier. This implies that under the same Brittlebench perturbations, LogiQA's conditional-reasoning items should lose more accuracy than its categorical-reasoning items. Ordinary item difficulty would make the opposite gap plausible to explain away, so we compare the drop for each reasoning type after controlling for clean accuracy.
The third check is internal. The misplaced-stretching index should peak at the layer where linear-probe readouts of the answer first diverge between clean and perturbed prompts. If no such layer exists, form edits act only at the final layer, which would tie brittleness to the readout rather than to the concept subspaces described in the Geometry paper.
References
- Information theoretic study of the neural geometry induced by category learning
- Brittlebench: Quantifying LLM robustness via prompt sensitivity
- POSIX: A Prompt Sensitivity Index For Large Language Models
- The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance
- Do Prompt-Based Models Really Understand the Meaning of Their Prompts?
- The Influence of Language Experience on the Categorical Perception of Vowels: Evidence from Mandarin and Korean
- Categorical Perception: A Groundwork for Deep Learning
- The Geometry of Categorical and Hierarchical Concepts in Large Language Models
- FACT: Learning Governing Abstractions Behind Integer Sequences
- ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models
- LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
- Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Data