Language models often look unbiased on standard social-bias tests, and then produce stereotyped answers once the same kind of question is made harder. Two explanations fit this pattern, and they have different consequences.
The first explanation is reasoning under load. When a model has more premises to track, it falls back on stereotypes as a shortcut, so the bias is a property of the model's reasoning. The second explanation is memorization. Popular bias benchmarks such as BBQ, a hand-built question-answering test of social bias, are public. Their items, or close relatives of them, have plausibly entered training and safety-tuning data. On that account, the neutrality we see on easy items is partly recalled, and "harder" items simply move the model off the memorized distribution.
Current dynamic evaluations cannot tell these apart, because every perturbation they apply makes an item both harder and new. This proposal builds a generator that varies the two factors independently. It then checks the resulting estimates against models that we contaminate on purpose.
Core Claim
Complexity-induced bias curves reported so far confound two quantities: the effect of reasoning load and the effect of leaving a memorized benchmark distribution. We propose a factorial design that crosses complexity level with surface provenance (original benchmark wording versus freshly generated but logically identical items). This design separates the two quantities, and the separation can be validated.
Let $B(p, c)$ be a bias score measured on items with provenance $p$ (original or fresh) at complexity level $c$. Complexity here means the number of reasoning steps and distractor premises. We define the contamination component at each complexity level as
$$\Delta(c) = B(\text{fresh}, c) - B(\text{orig}, c),$$
and the genuine complexity effect as the slope of $B(\text{fresh}, c)$ in $c$.
The memorization hypothesis predicts that $\Delta(0)$ is already positive: the model shows bias on novel easy items even though it looks neutral on the originals. The reasoning-load hypothesis predicts $\Delta(c)\approx 0$ at every level, with a rising slope. The distinction matters for mitigation. The first calls for evaluation hygiene. The second calls for changes to how models reason.
Grounding in Prior Work
Evidence that bias grows with complexity. Uncovering Stereotypes in Large Language Models (LSI) scales prompt complexity across seven task types. It finds that ChatGPT's refusal rate falls from 32.8% to 29.4% and that GPT-4's stereotype polarity becomes more spread out as complexity rises. DARG perturbs BBQ through reasoning graphs and reports that GPT-4 Turbo retreats to "Cannot be determined" as complexity rises.
Neither paper checks whether the unperturbed prompts were familiar to the model. DARG's perturbations always produce text the model has not seen, so memorization and difficulty move together. The two papers also disagree about the direction of failure: LSI finds more overt stereotyping, while DARG finds more evasion. Our metrics therefore track evasion and stereotyping separately.
The method we reuse, and where it stops. A Careful Examination of LLM Performance on Grade School Arithmetic built GSM1k, a set of fresh problems matched to GSM8k. It found drops of up to 8% for some model families and a modest link between low token surprise and the size of the drop (Spearman $r^2=0.36$). We borrow the idea of fresh isomorphic items, but apply it to social bias rather than arithmetic accuracy. We also cross it with complexity, which GSM1k does not vary.
DyVal supplies the idea of generating items from a symbolic graph with controllable depth and width, so that ground truth comes from the generator rather than from an LLM. DyVal covers only formal domains. DARG's LLM-based pipeline, by contrast, struggles to verify subjective social content.
Constraints on the design. ABCD shows that multiple-choice scores can reflect label and position artifacts rather than content: some models exceed 95% accuracy on meaningless NonsenseQA items. When Benchmarks are Targets shows that option shuffling reorders leaderboards. Any difference we measure could therefore be a formatting artifact, so we control format in every condition. The EHRSQL debiasing study (Towards Unbiased Evaluation) shows how surface cues inflate scores: F1 falls from 93.2 to 1.6 for an n-gram filter once those cues are removed. This is why our fresh items must match the originals' surface statistics rather than merely differ from them.
The regime we study. Leaderboard Incentives shows that tiny amounts of targeted post-training (18 steps on Winogrande) can move leaderboard rank. Safety tuning on public bias benchmarks is exactly such targeted training. The contamination survey notes that contamination inflates fairness metrics as well as accuracy, but it offers no controlled test of how.
Research Program
The item generator. We will build a generator that produces BBQ-style items from a symbolic "fact graph": who was present, who did what, and which premises disambiguate the answer. Each graph node is realized in English from a pool of templates disjoint from BBQ. The demographic attribute of each person is assigned by the generator and counterbalanced, so every item appears in a stereotype-consistent and a stereotype-conflicting version with identical syntax. We use the social categories and stereotype attestations from BBQ.
Complexity is raised in three ways: more inference steps between the disambiguating evidence and the question, more parallel distractor premises, and greater textual distance between a person's first mention and the question. The gold answer, including "unknown" for ambiguous contexts, is computed from the graph.
Paired items at matched complexity. For each original BBQ item we produce two complexity-matched counterparts:
- An "original-derived" item, which keeps the original sentences verbatim and embeds them in additional graph structure.
- A "fresh" item, with the same graph and statistics but new names, scenarios and wording.
Complexity level 0 of the fresh arm is effectively a BBQ version of GSM1k.
Validating the decomposition. We will fine-tune open-weight models, including families that GSM1k found to be overfit such as Mistral and Phi, on BBQ items at graded doses. We will also train an uncontaminated control. This mimics strategic safety post-training and gives known ground truth: the estimated $\Delta(0)$ should rise with dose while the fresh-item slope stays roughly unchanged. If it does not, the method is not measuring contamination, and we report that.
Frontier models. With the method validated, we apply the decomposition to API and open frontier models, including reasoning-tuned models. Curse of Knowledge found that reasoning models degrade more under biased context (DeepSeek-R1 declines 3.35% versus 1.27% for DeepSeek-v3). We ask whether that fragility persists on fresh items.
Baselines
Do nothing. The naive reference point is the model's score on the original BBQ test set, reported as developers usually report it. We measure the share of wrong ambiguous-context answers that are stereotype-aligned, where UnifiedQA reached roughly 77% in the original BBQ paper. We also measure the disambiguated-context gap between stereotype-conflicting and stereotype-aligned items, which was 3.4 percentage points in that paper. Our claim is that this number overstates neutrality for contaminated models. It is beaten if fresh level-0 items show a significantly larger gap for dosed models and for some public models.
Strongest existing method: DARG-style perturbation. DARG perturbs BBQ along graph depth and width. We will run its perturbations on the same BBQ seed items and models, with the same decoding settings and number of samples. The file reports only a qualitative BBQ finding (models defaulting to "Cannot be determined"). The only numeric figure is Claude-3-Opus losing over 50% accuracy when reasoning depth increases by four steps, and it is not stated for BBQ. So no reported BBQ result was found on Lacuna for DARG. We beat it if, on dosed models with known contamination, our decomposition recovers the dose-response in $\Delta(0)$ while DARG's single complexity curve attributes the dose effect to complexity.
Perplexity-surprise detection. The main contamination baseline flags items by low token surprise, following GSM1k. We add ABCD's comparison of perplexity under original versus permuted option order. These detectors run on identical items. We beat them if our behavioral $\Delta(0)$ correlates with injected dose more strongly than surprise does. The comparison anchor is the $r^2=0.36$ reported in the GSM1k study.
Ablations. The first ablation removes the fresh-provenance arm and collapses the design to complexity-only perturbation. This should reproduce the confound. The second holds complexity at level 0 and varies provenance only. It tests whether the complexity axis adds anything beyond a GSM1k-style fresh set.
Why not just…?
Why not just standardize the prompt format? Formatting artifacts are real. ABCD's Matched-and-Dashed protocol, which replaces option letters with dashes and matches answers by embedding, cuts accuracy variance across option permutations threefold. We therefore evaluate every condition under that protocol and also under free-form answers. If $\Delta(c)$ and the complexity slope both vanish under the protocol, the phenomenon was formatting and we will say so. DARG's report of large depth-driven drops under fixed prompt style suggests it will not vanish.
Why not just run perplexity or n-gram checks on BBQ? These methods tell you whether text is familiar, not how much familiarity inflates the fairness score. They also miss paraphrase and cross-lingual leakage (Data Contamination Can Cross Language Barriers). The GSM1k study documents an RL-trained model that was overfit despite a low memorization signal, and bias benchmarks likely leak through safety-tuning data in paraphrased form. The dose experiment directly compares these detectors against our behavioral measure.
Why not just build a fresh BBQ, GSM1k-style, and skip the complexity axis? This is the most serious threat. It answers the question "is BBQ neutrality inflated?" more cheaply, and our level-0 fresh arm is exactly this study. What it cannot answer is whether published complexity-bias curves reflect reasoning or memorization. Answering that requires the interaction between complexity and provenance.
If the dosed experiments show that $\Delta(c)$ is constant across complexity, the cheap design suffices for contamination questions, and we will report that. What remains for our design is the slope measured on fresh items, as a clean estimate of complexity-induced bias.
Why not just call it syntactic degradation? Our items differ only in which attribute is attached to which role, and syntax is identical. So any gap between stereotype-consistent and stereotype-conflicting versions cannot be a parsing failure. WinoMT used the same logic to show 80% versus 54% accuracy on stereotypical versus non-stereotypical roles in French. BUG attributes about 23% of its errors to genuine ambiguity, which we exclude by construction because every gold label is fixed by the graph.
Decisive Checks
The first check is the dose-response. In deliberately contaminated models, $\Delta(0)$ must grow with the number of injected BBQ items while the control's stays near zero. Without this, nothing else is interpretable.
The second check covers public models: whether $\Delta(0)$ is reliably positive, and whether the fresh-item slope is flatter than the slope on original-derived items. A positive answer would mean existing complexity-bias curves overstate reasoning failure.
The third check separates the two ways of failing. For disambiguated items, we measure evasive "unknown" answers and stereotype-aligned errors separately. This tests whether DARG's evasion pattern and LSI's overt-stereotype pattern come from different provenance conditions rather than from different models.
The fourth check compares reasoning-tuned models with their base models on fresh items only. It asks whether their extra fragility survives once memorization is removed.
References
- Uncovering Stereotypes in Large Language Models: A Task Complexity-based Approach
- DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic
- DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
- ABCD: All Biases Come Disguised
- When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
- Towards Unbiased Evaluation of Detecting Unanswerable Questions in EHRSQL
- Leaderboard Incentives: Model Rankings under Strategic Post-Training
- Benchmark Data Contamination of Large Language Models: A Survey
- BBQ: A hand-built bias benchmark for question answering
- Curse of Knowledge: Your Guidance and Provided Knowledge are biasing LLM Judges in Complex Evaluation
- Data Contamination Can Cross Language Barriers
- Evaluating Gender Bias in Machine Translation
- Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation