When a reasoning model is fine-tuned on a fixed set of math problems for several passes, it often gets worse on new problems. In the simplest theoretical setting this is proven: revisiting training examples makes stochastic gradient descent memorize them. A common response is to have a large language model (LLM) rewrite each problem into many variants. The rewrites might change names, numbers, the story, add distractors, or change the problem's structure. The unanswered question is whether a rewritten problem acts like a genuinely new training example or like a disguised second pass over the old one.
We propose to measure this directly. The measurement is an exchange rate: how many fresh, independently drawn problems each augmentation family is worth. We run it in settings where a procedural generator can supply unlimited contamination-free fresh problems as ground truth. We then test whether a cheap signal can predict that exchange rate without the expensive sweep. The signal is the similarity between a variant's training gradient and its source problem's gradient. If it can, practitioners working in domains without a generator could tell in advance which augmentations buy real generalization and which only prolong memorization.
Core Claim
Our hypothesis is that LLM augmentations lie on a continuum between "repeat the example" and "draw a fresh example." Where a family sits depends on whether it changes the reasoning the model must execute, not on how different the text looks. Paraphrase and narrative rewrites should behave mostly like extra passes. Number resampling, added reasoning steps, and changes to the symbolic structure should behave mostly like fresh data. We also claim that the similarity between a variant's gradient and its seed's gradient, measured early in training, predicts a family's position better than text-embedding distance does. Embedding distance is the diversity proxy now used in the augmentation literature.
To make this precise, suppose training starts from $n$ seed problems, and each seed is expanded into $k$ variants by augmentation family $A$. Let $n_{\text{eff}}(A,k)$ be the number of fresh problems from the generator that reaches the same held-out accuracy under the same training budget. We define the freshness of $A$ as
$$\rho_A(k) = \frac{n_{\text{eff}}(A,k) - n}{n(k-1)},$$
so $\rho_A = 0$ means the variants were worth no more than repetition, and $\rho_A = 1$ means each variant was as good as a new problem. Negative values mean the augmentation actively hurt. That is plausible, because DéjàQ reports that plain resampling of the original data drove Qwen2.5-7B-Instruct from 62.6% down to 46.4% on GSM-P2.
Why This Gap Is Real
Three lines of work motivate the question.
The first supplies the optimization side. Rapid Overfitting of Multi-Pass SGD in Stochastic Convex Optimization proves that one pass of SGD achieves optimal population risk, while a second pass over the same data can push excess risk to $\Omega(1)$. The gap between those two regimes is exactly what augmentation is supposed to close. That result concerns exact repetition in convex models, however, and says nothing about near-duplicates. Our freshness measure fills that interval empirically. Older evidence points the same way: Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping found that faster fitting of the same data worsens generalization.
The second line shows why reasoning is the right place to test this. GSM-Symbolic shows that changing only names and numbers moves accuracy by up to 12% for Gemma2-9b, and that one irrelevant clause dropped Phi-3-mini by 65%. This is evidence that models attach to surface features that surface-level augmentation would keep constant. Reasoning or Memorization? constrains our design. It shows that gains on MATH-500 for Qwen2.5 can reflect pretraining leakage: Qwen2.5-Math-7B scores nearly 40% from only the first 40% of each problem. It also provides RandomCalculation, a leakage-free generator. We therefore treat generator-backed, contamination-free evaluation as the primary measurement and standard benchmarks as secondary.
The third line supplies the augmentation methods whose value we want to price. DéjàQ evolves problems during reinforcement learning with setting, distractor, and symbolic mutators, and selects the ones a student solves only some of the time. Adaptive Problem Generation via Symbolic Representations (Opt-Sym) mutates problems as SymPy programs. It reports much higher embedding diversity than natural-language rewriting, with pairwise cosine distance 0.37 versus 0.07. Both report benchmark gains. Neither asks how much of that gain fresh data of the same budget would have delivered, and neither separates "more diverse text" from "different reasoning." We reuse their mutators as the augmentation families under study. We do not propose a new generator.
Research Program
The core experiment is an exchange-rate sweep in two generator-backed domains. The first is GSM-Symbolic templates, which give fresh instances at a fixed structure, plus P1 and P2 variants with added steps. The second is RandomCalculation, which gives fresh multi-step arithmetic of controllable length. In each domain we first draw fresh-data learning curves: held-out accuracy after training on $m$ fresh problems for $m$ ranging over roughly two orders of magnitude, always at a matched number of optimizer steps. This curve is the ruler.
We then fix $n$ seeds and expand them with $k \in \{2, 4, 8, 16\}$ variants from each of five families. The families are LLM paraphrase, narrative-setting rewrite, number resampling with recomputed answers, distractor insertion, and structural mutation that adds or changes operations. The last three follow DéjàQ's mutators and Opt-Sym's symbolic edits. Each augmented set is trained at the same step budget, and its accuracy is read off the fresh-data curve to obtain $n_{\text{eff}}$ and $\rho_A(k)$.
Training uses group relative policy optimization (GRPO), a reinforcement-learning method that rewards verified-correct sampled solutions relative to other samples for the same prompt. We use it because it is the regime DéjàQ and DaGRPO study. We repeat the sweep with supervised fine-tuning on verified solutions to check whether freshness depends on the objective. Models are Llama3.1-8B and Qwen2.5-7B-Instruct. We include Llama because Qwen's contamination could inflate any method's gains on public benchmarks.
The second component is prediction. At an early checkpoint we compute two quantities for each family. One is the average cosine similarity between the gradient of a variant's loss and the gradient of its seed's loss, using per-example gradients on a random low-dimensional projection to keep cost manageable. The other is the embedding cosine distance used by Opt-Sym. We test which quantity rank-orders the families' measured $\rho_A$, and whether a family's gradient similarity predicts its $\rho_A$ when that family is held out of the fit.
As a secondary diagnostic, we track the largest Hessian eigenvalue at convergence, as used in Towards Generalizable 3D Human Pose Estimation via Ensembles on Flat Loss Landscapes. This tests whether low-freshness augmentation also leaves the model in sharper minima, the way extra passes are expected to. We treat this as an explanatory probe, not the main result.
Baselines
The naive reference point is repetition. The same $n$ seeds are trained for $k$ passes at the matched step budget, with no augmentation. This defines $\rho = 0$ by construction and shows how much damage multi-pass training does in each domain. The opposite anchor is one pass over $nk$ fresh generator problems, which defines $\rho = 1$. Both anchors use identical models, step counts, learning-rate schedules, and hyperparameter search. Each condition gets the same small grid over learning rate and number of GRPO samples per prompt, and results are selected on a separate fresh validation split.
The strongest existing augmentation method is DéjàQ with all mutators and learnability-based selection. It is the strongest competitor because it already combines the structural mutators we study with adaptive selection. It is also the only method in the file that reports beating both the base model and resampling on GSM-P2. DéjàQ reports 65.5% on GSM-P2 for Qwen2.5-7B-Instruct, against 62.6% for the base model and 46.4% for resampling. On MATH-500 it reports 69.6% against a base of 68.0% (DéjàQ). We rerun DéjàQ with the same teacher model, prompt budget, and step count as our conditions, and compute its $\rho$ on the fresh-data curve.
Opt-Sym serves as the symbolic-generation baseline. It reports a 3.3% average accuracy improvement for Qwen2.5-1.5B across GSM8K and MATH-500 (Adaptive Problem Generation via Symbolic Representations). We reuse its SymPy mutation step, but with an open teacher, so that no condition depends on a proprietary generator.
This proposal does not claim to beat DéjàQ's accuracy. Its claim concerns prediction, and "beating" is defined accordingly. The gradient-similarity signal must rank-correlate with measured $\rho_A$ significantly better than embedding distance does, across families and both model families. In addition, an augmentation budget chosen by our signal must match DéjàQ-level fresh-data accuracy while spending fewer teacher generations.
The key ablation isolates the signal from the family label. Within the single family "structural mutation," we sort individual variants by gradient similarity and train on the lowest and highest tertiles separately. If low-similarity variants are worth more fresh data than high-similarity variants from the same mutator, the signal carries information beyond which mutator produced the variant.
Why not just…?
Why not just generate fresh problems procedurally and skip augmentation entirely? Where a generator exists, one should, and our fresh-data anchor confirms it. But generators exist only for narrow domains. Adaptive Problem Generation via Symbolic Representations notes that symbolic formalization excludes informal and abstract reasoning. The point of pricing augmentation in generator-backed domains is to calibrate a signal that can be used where no generator exists. The honest risk is that the exchange rates we measure will not transfer to other domains. We test transfer by checking whether the predictor's ranking holds on ConsisEval's code pairs, where no generator is used for evaluation.
Why not just use early stopping or sharpness-aware minimization on repeated data? These are real threats. If early stopping on repeated seeds matched augmented training, augmentation would be pointless. We run repetition with validation-based early stopping, and repetition with sharpness-aware minimization, as additional anchors. The multi-pass theory (Rapid Overfitting of Multi-Pass SGD) suggests that stopping only caps the loss from repetition and cannot add information. Our fresh-data curve measures whether that holds for LLMs: if early-stopped repetition reaches $n_{\text{eff}}$ well above $n$, the claim fails.
Why not just measure text diversity with embeddings, as Opt-Sym does? This is the cheapest competitor to our predictor and it may work. Opt-Sym's 0.37 versus 0.07 gap already separates symbolic from linguistic edits. Our concern is distractor insertion. It makes text look very different while leaving the required computation unchanged, and GSM-Symbolic's no-op results show that models react strongly to exactly such edits. The head-to-head rank-correlation test decides this. If embedding distance predicts $\rho_A$ as well as gradient similarity, we report that the cheap proxy suffices, and the contribution becomes the measured exchange rates themselves.
Why not just use DéjàQ's learnability filter? Learnability selects problems of the right difficulty, but it does not distinguish a hard near-duplicate from a hard new problem. The within-family tertile ablation, run after learnability filtering, tests whether freshness adds value on top of difficulty selection.
Decisive Checks
The primary metric is accuracy on fresh generator draws that never appear in training, reported alongside $\rho_A(k)$ with confidence intervals from three seeds. Secondary metrics are GSM-P1, GSM-P2, and GSM-NoOp accuracy, RandomCalculation accuracy at longer lengths than those used in training, and MATH-500 accuracy for Llama3.1-8B only. The contamination audit in Reasoning or Memorization? makes MATH-500 unreliable for Qwen. As an exploratory measure, we report the expansion rate on MATH-B-U from MATH-Beyond, the fraction of problems solved after training that the base model failed across 1,024 samples, to see whether high-freshness augmentation expands capability rather than only sharpening existing solutions. There, the best reinforcement-learning expansion rates reported are 5.22% for DeepScaleR-1.5B and 21.2% for Skywork-OR1-7B.
Three outcomes would falsify the core claim. The first is if every family, paraphrase included, lands near $\rho = 1$, meaning any rewrite counts as fresh data. The second is if structural mutation lands near $\rho = 0$, meaning the model treats even computation changes as repetition. The third is if gradient similarity fails to beat embedding distance at ranking families. Any one of these would still yield a useful, publishable exchange-rate table. Only a clean separation, with surface families near repetition and structural families near fresh data, predicted by gradient similarity, would support using the signal to budget augmentation in domains without generators.
References
- Rapid Overfitting of Multi-Pass SGD in Stochastic Convex Optimization
- Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
- Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
- DéjàQ: Open-Ended Evolution of Diverse, Learnable and Verifiable Problems
- Adaptive Problem Generation via Symbolic Representations
- Towards Generalizable 3D Human Pose Estimation via Ensembles on Flat Loss Landscapes
- MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model