When a Simpler Wrong Rule Wins: Description Length as a Predictor of In-Context Distraction

Large language models can learn a new task from a few worked examples placed in the prompt, a capability called in-context learning. They are also easily thrown off by irrelevant or misleading text in that prompt. Most work treats this distraction as a matter of quantity, where more junk tokens dilute the model's attention. We propose that, for in-context learning, the damage comes from something more specific. A distractor hurts when it makes some wrong rule a shorter explanation of the demonstrations than the true rule. The amount, length, or randomness of the noise matters much less. Models are known to prefer simple hypotheses. A spurious feature, such as "every positive example mentions a city," therefore wins whenever it is simpler to state than the intended task. The same feature should be ignored when it is more complicated than the task.

The claim can be tested, and it can be wrong. If it holds, it changes how prompts should be cleaned. Compressing the text, as gzip-based or summarization-based filtering does, should matter little. What should matter is finding the shortest wrong rule the demonstrations support and adding a counterexample that rules it out. We test the claim on controlled demonstration sets. Language models draft them and human reviewers verify them, so that rule complexity can be varied while length and lexical overlap stay fixed. We then use the model's internal "task vector" to check which rule it actually adopted. Finally, we turn the finding into a demonstration-repair procedure and compare it against the strongest selection method in the file.

Core Claim

The claim has three parts. First, it measures complexity on the rule, not the text. Kolmogorov complexity is the length of the shortest program that produces an object; in practice it can only be bounded from above, usually by a compressor. We approximate it for the mapping from demonstration inputs to labels: the length of the shortest program, in a small rule language, that reproduces the labels. Second, it predicts when the model switches rules. Suppose the model behaves like a Bayesian learner with a prior proportional to $2^{-K(h)}$, where $K(h)$ is the description length in bits of hypothesis $h$. Let $\Delta K = K(h_{\text{true}}) - K(h_{\text{spur}})$ be the extra bits needed to state the true rule over the spurious one. Let $D$ be the average evidence, in bits, that one demonstration provides against the spurious rule. Then the number of demonstrations needed before the model abandons the spurious rule is roughly

$$ n^* \approx \frac{\Delta K}{D}, $$

because each demonstration adds about $D$ bits to the log posterior odds and the prior starts $\Delta K$ bits behind. When the demonstrations contain no evidence against the spurious rule ($D = 0$), the spurious rule should persist however many examples are added. Third, the claim names its own falsifier. If error rates track text compressibility, token count, or lexical overlap with the query, and not $\Delta K$, the hypothesis is wrong.

Grounding in Prior Work

Two papers motivate the claim. Bigelow and colleagues frame in-context learning as Bayesian model selection over rules. In In-Context Learning Dynamics with Random Binary Sequences and Subjective Randomness and In-Context Learning, they show that models switch hypotheses abruptly as evidence accumulates. They also show that models prefer shorter descriptions, measured with gzip on generated binary strings, and that these dynamics appear mainly in instruction-tuned models. Their work stays within binary coin-flip sequences and never asks what happens when the prompt contains a competing explanation. We carry their account into natural-language classification and reasoning, and we make the competing hypothesis the object of study. In In-Context Learning Creates Task Vectors, Geva and colleagues show that the demonstrations are compressed into a single intermediate-layer vector. Patching that vector into a query-only forward pass restores 80–90% of in-context accuracy. They tested only clean prompts. We reuse their patching method as a readout of which rule the model encoded under contamination.

Three papers constrain the design. Thinking in a Crowd shows that misleading snippets drop Qwen3-8B from 96.22% to 77.03% accuracy on SciAux even with step-by-step reasoning enabled. This tells us that reasoning modes do not remove the problem and must be included as a condition. Unifying Demonstration Selection and Compression for In-Context Learning (UniICL) finds that strong compression of demonstrations helps by "filtering noise," and that lexically similar retrieved demonstrations mislead. Our account offers a reason: lexical similarity is a cheap way to introduce short spurious rules. Finally, Less is More shows that human summaries of programming problems beat machine summaries, at +11.88% on CodeContests. That result points toward a text-shortening explanation, and our design tests that explanation against the rule-complexity one.

Two papers supply methods for building the data. Human verification is what makes the controlled sets trustworthy. INSPECTOR shows that automated text transformations often corrupt labels unless humans audit them. Review of model-drafted data is about 13 times cheaper than writing from scratch, per Building a Role Specified Open-Domain Dialogue System. ToolQA shows that templates with programmatic checks give verifiable ground truth. We combine these: templates fix the rules, models draft the surface text, and humans review it.

Research Program

The first component is a controlled benchmark. We take base tasks from TREC, SST-2, and AG News, which are the classification sets used by Submodular Context Partitioning. For each task we write the true rule and a family of candidate spurious rules in a small rule language. The rule language has atoms such as "mentions a place name," "has fewer than k words," and "contains token w," joined by and, or, and not. A rule's description length is its encoded program length in this language. As a sanity check, we also compute the gzip size of the rule string.

A language model drafts demonstrations that satisfy both the true rule and a chosen spurious rule. It also drafts matched controls, which contain the same distractor content arranged so that it does not line up with the labels. Human reviewers check three things for each set: the text really satisfies both rules, there is no third shortcut, and the label is right. The design is factorial over three axes: $\Delta K$; distractor token count (held fixed or varied); and lexical overlap between distractor and query (BM25, held fixed or varied).

A reasoning extension uses SciAux. Here reviewers annotate which misleading snippets imply a simple wrong conclusion and which imply a convoluted one.

The second component is mechanistic. On LLaMA-7B and Qwen3-8B, in both base and instruction-tuned variants where available, we extract task vectors from contaminated prompts. We patch each vector into probe queries on which the true and spurious rules disagree. The probe accuracy tells us which rule the vector encodes. We then sweep the number of counterexamples to measure where the model flips, which gives an empirical $n^*$, and compare it with $\Delta K / D$.

The third component is a repair procedure. We enumerate the shortest spurious rules consistent with a given demonstration set by searching the rule language in order of length. A model then drafts a minimal counterexample that breaks the shortest one, and a human verifies it. The verified counterexample replaces the least informative demonstration, so the shot count stays fixed. The procedure needs no training and touches only the demonstrations.

Baselines

The naive reference point uses randomly sampled demonstrations with the distractor left in. It also includes a matched clean condition with no distractor, which serves as the upper reference.

The strongest existing selection method is Sub-CP, from Submodular Context Partitioning and Compression for In-Context Learning. It chooses demonstrations by maximizing a facility-location diversity objective. It is the strongest competitor because diverse selection might break spurious correlations by accident. The paper reports a +29.2% absolute accuracy gain over random selection on TREC with Local Diverse partitioning and Product-of-Experts ensembling. It also reports ICAE compression accuracy rising from 60.92% to 66.26% with Global-Local Diverse partitioning. We run it with its published configurations, on the same candidate pools, models, and shot counts as our method. Our method beats it if repair exceeds Sub-CP's accuracy on spurious-aligned probe queries, averaged over three seeds and with non-overlapping confidence intervals, while not losing accuracy on queries where the two rules agree.

The prompt-level instruction baseline adapts the negation prompt from Distraction-aware Answer Selection in Improving Zero-shot Reader by Reducing Distractions. That method reports a 64% average Exact Match gain on open-domain question answering, a different setting from ours. Here we tell the model to ignore irrelevant features and give it the same budget for prompt tuning.

Three ablations isolate the key ingredient. The first adds the same number of human-verified counterexamples, chosen at random rather than aimed at the shortest spurious rule. The second aims at the second-shortest spurious rule. The third aims at the shortest rule but skips human verification. If random counterexamples match targeted ones, description length is not doing the work.

For the analysis claim, the competing predictors are distractor token count, gzip compressibility of the demonstration text, model perplexity of the distractor, and BM25 overlap. All are fit in the same regression with $\Delta K$, and the comparison is by held-out explained variance.

Why not just…?

Why not just compress or filter the demonstrations by gzip size or perplexity? This is the obvious move, and it measures the wrong object. A short, highly compressible demonstration can carry a strong spurious rule. A long, noisy one can carry none. Our factorial design holds text length fixed while varying $\Delta K$, which separates the two directly. If text compressibility predicts errors as well as rule complexity does, we drop the rule-level account and report that outcome.

Why not just use diversity-based selection such as Sub-CP? This is a real threat, because diversity often breaks shortcuts as a side effect. The difference is that diversity is computed without regard to which wrong rule is cheapest. Sub-CP's own paper notes it is query-blind and computes nothing about description length. We predict it will help on high-$\Delta K$ spurious rules, which random variation tends to break, and fail on the shortest ones, which survive diverse sampling. If it matches targeted repair across the $\Delta K$ range, the repair component offers nothing new. The analysis findings would still stand.

Why not just fine-tune for robustness, or turn on step-by-step reasoning? Fine-tuning on mixed relevant and irrelevant contexts works for retrieval, as Making Retrieval-Augmented Language Models Robust to Irrelevant Context shows with gains of over 30 points on 2WikiMQA. But it changes the weights and targets irrelevant passages, not label-aligned shortcuts inside demonstrations. We include a LoRA model trained that way as a reference condition. Reasoning mode is directly contradicted by the SciAux drop from 96.22% to 77.03%. We still run thinking on and off to test whether $\Delta K$ predicts which misleading snippets cause the rationalization failure that paper describes.

Proposed Experiments and Decisive Checks

The first decisive check is the regression. On held-out task–distractor pairs, $\Delta K$ must explain substantially more variance in spurious-rule adoption than token count, text compressibility, or lexical overlap. This must hold in instruction-tuned models. Bigelow's results predict weaker effects in base models, and we test that prediction too.

The second check is the switch point. The empirical number of counterexamples needed to flip the model should grow with $\Delta K$, roughly linearly at fixed per-example evidence, as the equation above predicts. It should not be flat. The task-vector readouts must also agree with the behavioral outputs on which rule was adopted.

The third check is the repair result on TREC, SST-2, AG News, and SciAux. Targeted counterexamples must beat random counterexamples and Sub-CP on spurious-aligned queries at equal shot count. The human-verified variant must beat the unverified one by a margin that justifies the reviewer time, which we log per item.

A failure of the first check refutes the core claim. A failure of only the third leaves a measurement result without a usable method, and we would report it that way.