Is the Synthetic-to-Real Gap Mostly a Noise Gap?

Models trained on made-up reasoning data transfer surprisingly well to real questions, yet they reliably stay below models trained on real data. We propose to test a specific explanation for that remaining gap. Synthetic training data is clean, while real documents are cluttered. A clinical note contains yesterday's creatinine next to today's, a relative's history next to the patient's, and pages of narrative around the one number that matters. If the gap comes from clutter, a model can learn logic from fictional data and still fail because it never practiced picking the right fact out of a crowd of plausible wrong ones. The practical test is simple. Generate fictional training worlds whose clutter matches the statistics of a real target domain, then check whether this closes the gap on the specific errors that clutter causes.

We study this on clinical calculation, where a model reads a messy patient note, pulls out the right values, and computes a score. Final accuracy there hides which of those steps failed, so we also build a domain-specific scoring method. It assigns each wrong answer to one of three causes: grabbing a distractor value, retrieving the wrong formula, or making an arithmetic mistake. The contribution is a way of splitting a synthetic-to-real transfer gap into a noise part and a reasoning/knowledge part, and a training recipe whose success or failure on the noise part is directly measurable.

Core Claim

The claim is that most of the synthetic-to-real shortfall in noisy expert domains is a "noise tax." We define the noise tax as the drop in accuracy between giving the model only the relevant sentences of a note and giving it the full note. We further claim that this tax can be reduced with fictional training data that contains confusable distractors of the same type as the targets, such as other values of the same variable at other times or values of similar-looking variables. Generic filler text of the same length should not reduce it. If this holds, the tax should shrink under matched-distractor synthetic training while accuracy on the clean, relevant-sentences-only version of each note stays the same. The shrinkage should also concentrate in distractor-capture errors rather than arithmetic or formula errors. If the tax does not move, the explanation is wrong and the ceiling must come from missing knowledge or format.

How this relates to existing work

Several papers motivate the claim. Learning from Synthetic Data Improves Multi-hop Reasoning trains small Qwen3 and Phi-4 models on purely fictional, grammar-generated worlds using a reinforcement learning method called GRPO. It reports 56–131% relative F1 gains on real multi-hop QA, but it trains and evaluates on clean text, scores with token F1, and notes a persistent real-data ceiling. We reuse its fictional-world design and its reinforcement learning with checkable rewards. We change what the fictional world contains and how transfer is scored.

CogniLoad controls distractor density independently of task length and difficulty. It finds a U-shaped response to intermediate density, but it uses these parameters only to probe models, never to train them. We use its density parameter as a training knob.

MedCalc-Bench supplies both the target task and the evidence that extraction amid narrative is a main failure point. GPT-4 reaches only 50.9% with one-shot chain-of-thought and 20.8% zero-shot. That benchmark never varies noise, and it scores only final answers.

Other work shows what does and does not help with noise. Huang et al. (Breaking Focus, Adaptive Distraction) show that a single coherent distracting sentence cuts accuracy by more than 45% and that "ignore irrelevant text" prompts barely help. They also show that preference training on distracted-versus-clean pairs recovers a large share of that loss, nearly 50% in some small models. Ext2Gen trains an extract-then-answer behavior on real noisy retrieval and beats separate compressors (0.557 vs 0.473 for Llama3.1-8B). Both methods train on real, in-domain noise. Our question is whether fictional noise with no domain knowledge in it can substitute for real noise. This matters wherever real annotated notes are scarce or private.

AbstRaL removes distractor-induced drops on GSM-Plus by converting word problems to symbolic form. It is confined to math, where abstraction is natural. Clinical notes do not reduce cleanly to equations before the values are extracted.

One paper constrains the design. Breaking Barriers finds that reinforcement post-training on math and code gains 2.87% in-domain but loses 3.19% out-of-domain, including on MedQA and LegalBench, and that larger models overfit more. Any structured-data reinforcement learning we do could inherit this damage, so we monitor it explicitly. Fictional data may avoid it, because it contains no domain facts to overfit to. That remains to be shown.

Research Program

Building the fictional worlds. We will build a generator of fictional clinical-calculation worlds. It creates fictional patients and invented lab analytes with made-up names and units. It also creates invented scoring rules, both formula-based and point-based, mirroring MedCalc-Bench's split between equation tasks and rule tasks. Every quantity is computable, so rewards are exact. Crucially, no real medical fact appears.

Measuring real clutter. On a held-out development slice of real notes, we will annotate the evidence spans for each required parameter in a stratified sample of several hundred MedCalc-Bench instances. From these annotations we estimate a clutter profile with four parts:

  • the ratio of relevant to irrelevant sentences;
  • how often the same variable appears at multiple time points;
  • how often confusable variables co-occur;
  • how often units must be converted.

Training conditions. The generator then produces four matched corpora with identical reasoning content and identical token length:

  • clean;
  • padded with irrelevant fictional narrative (length only);
  • seeded with confusable same-type distractors matched to the real clutter profile;
  • seeded with adversarially searched distractors in the style of Huang et al.

We train Qwen3-scale models (0.6B to 7B, following the synthetic multi-hop study) with GRPO on each corpus under a fixed compute budget.

The error-attribution metric. For every prediction on the annotated subset, the model's chain of reasoning is parsed for the extracted parameter values. Each wrong answer is then assigned to one of three categories:

  • distractor capture: the extracted value appears elsewhere in the note but belongs to another time point or variable;
  • formula error;
  • arithmetic error, which is scored within the benchmark's 5% tolerance for equation tasks.

Final accuracy uses the benchmark's own rules: exact match for rule tasks and 5% tolerance for equation tasks. The noise tax is computed per model as accuracy on relevant-sentences-only notes minus accuracy on full notes.

Baselines

The naive reference points come first. The base model is evaluated zero-shot and with one-shot chain-of-thought on MedCalc-Bench. For scale, the benchmark reports GPT-4 at 50.9% (one-shot chain-of-thought) and 20.8% (zero-shot direct), and Llama 3-70B at 39.5% one-shot, all from MedCalc-Bench. Our small models will sit well below these numbers. The comparison of interest is between training conditions at a fixed model size, not against GPT-4.

The clean-synthetic GRPO model is the direct analogue of Learning from Synthetic Data Improves Multi-hop Reasoning moved to our fictional clinical worlds. It is the natural control, because the only thing that differs from our proposal is the clutter in the training data. Beating it means a lower noise tax at equal clean-note accuracy.

The length-matched filler condition is the ablation that isolates our key ingredient. If confusable same-type distractors matter, matched-distractor training must beat filler training on the distractor-capture error rate. Merely seeing longer contexts should not explain the gain.

The strongest existing competitors train noise robustness on real data:

  • preference training on real distracted-versus-clean MedCalc-style pairs, following Huang et al.;
  • an Ext2Gen-style model that extracts evidence and then answers, trained on real notes mixed with irrelevant sentences, following Ext2Gen.

These methods see in-domain noise that our method never sees, so they are the fair upper bar for noise handling. No reported result was found on Lacuna for either method on MedCalc-Bench. The nearest reported figures are Ext2Gen's 0.557 against 0.473 on noisy general QA and Huang et al.'s gain of nearly 50% on perturbed accuracy.

All conditions get the same base model, the same number of training tokens and optimizer steps, and the same hyperparameter search budget. They are evaluated on the same held-out split, disjoint from the notes used to estimate the clutter profile and to build real training pairs. We count it as success if matched-distractor synthetic training comes within a few points of the real-noise methods on full-note accuracy, and within that margin on the distractor-capture rate, while using zero real notes for training. We also report a hybrid condition that combines synthetic pretraining with a small real-noise set, to test whether the two are complementary.

Why not just…?

Why not just tell the model to ignore irrelevant text? Huang et al. test exactly this instruction and find almost no protection against coherent distractors, with drops still above 45% (Adaptive Distraction). MedCalc-Bench's zero-shot chain-of-thought results stay low. We still include an "ignore distractors" prompt as a cheap arm, because it costs nothing.

Why not just do preference training on real distracted examples? This is the real threat. Huang et al. report large recoveries this way, and Ext2Gen shows in-generation extraction training works on real noisy retrieval. If real-noise training beats fictional matched-noise training by a wide margin at equal compute, our recipe is merely a fallback for low-data settings. The noise-tax decomposition would still stand as a contribution. The baseline comparison and the hybrid condition settle which of these outcomes holds. We also test data efficiency by varying the amount of real noisy data from zero upward. This locates the point at which real data overtakes synthetic data.

Why not just extract the relevant sentences first and then compute? A two-stage pipeline would remove the noise before reasoning begins. Its ceiling is our relevant-sentences-only evaluation. Ext2Gen found that separate compressors (Recomp, CompAct, EXIT) underperform joint training. Clinical distractors are also unusual: they are the same variable at a different time, which a relevance filter can easily keep. We run a retrieve-then-compute pipeline with the same base model as a baseline and score it with the same error attribution. If the distractor-capture errors simply move into the retriever, that result confirms the problem is choosing the right value, not trimming irrelevant text.

Why not just train on more clean synthetic data, or use a bigger model? Breaking Barriers reports that larger models overfit their training domain more. CogniLoad shows 32B–70B models collapsing under length and difficulty. We double the clean corpus and compare two model sizes. If the noise tax does not shrink with clean scale but does shrink with matched clutter, scale is not the answer.

Decisive Checks

The central test compares clean and matched-distractor synthetic training. Relevant-sentences-only accuracy should be statistically indistinguishable between them. The noise tax and the distractor-capture rate should be lower for the matched condition, and lower still than for the filler condition. If the noise tax is equal across all synthetic conditions, the hypothesis is falsified.

A dose-response check trains at several distractor densities. If transfer tracks the real clutter profile, training density near the estimated real ratio should do best. That would turn CogniLoad's U-shaped evaluation curve into a claim about training.

Two transfer checks follow. We evaluate on GSM-Plus (Distract) against AbstRaL's setting, to see whether clinical-style clutter training generalizes to math distractors. We also run the MedQA, PubMedQA and LegalBench subset of the Breaking Barriers suite, to test whether fictional-data reinforcement learning avoids the roughly 3% out-of-domain loss reported for math and code training.

Finally, we audit the error attribution itself. Two annotators label a sample of attributions, and we report agreement. This matters because RadGraph shows that even expert agreement on clinical relations can be modest (kappa 0.397), and our metric is only useful if its categories are reliable.