Many recipes for improving a language model's reasoning without human labels use self-training. The model solves a pool of unlabeled problems many times, keeps the answers it agrees with most strongly, and fine-tunes on those as if they were correct. These kept answers are called pseudo-labels, and the step that decides which to keep is the filter. Work on semi-supervised learning has shown that confidence-based filters are fragile, because networks are often confidently wrong. Work on benchmark contamination, meaning test questions that leak into training data, has shown something that bears directly on these filters. When a model has memorized a problem, it answers that problem with unusual certainty and along an unusually narrow, repeated chain of reasoning. Put together, these findings predict that the standard self-training filter will preferentially select leaked benchmark items, the problems the model has already memorized. If so, self-training would quietly amplify contamination: it would spend its training budget reinforcing memorized shortcuts and inflate scores on exactly the benchmarks used to measure progress.
This note proposes to test that prediction under controlled contamination and to build a filter that fixes it. The filter separates two signals that current pipelines merge into one. The first is agreement on the answer: the model should reach the same answer when the problem is reworded or reshuffled. The second is diversity of the path: the model should reach that answer by varied reasoning across samples. Genuine reasoning should show high answer agreement under rewording together with varied paths. Memorization should show agreement that breaks under rewording together with collapsed, near-identical paths. The filter keeps a pseudo-label only when both signals point to genuine reasoning. It reuses the samples that majority voting already draws, so it adds only a few reworded prompts per problem.
Core Claim
The uncertainty-filtering literature treats low uncertainty as a sign that a pseudo-label can be trusted. In Defense of Pseudo-Labeling (UPS) keeps pseudo-labels with low variance across Monte Carlo dropout passes, which means running the network several times with random units switched off. Tri-net keeps predictions that stay stable across such passes, and USDNL shows a single dropout pass is enough to select clean samples under label noise. The contamination literature shows that, for memorized items, low uncertainty is a symptom rather than a guarantee. Detecting Data Contamination from RL Post-training finds that reinforcement learning collapses the model onto a narrow, deterministic path for problems it was trained on. Detecting RLVR Training Data via Structural Convergence finds that independent samples on seen prompts are nearly identical in structure. A Narrowing Geometry in Contaminated Reasoning finds that internal activations on leaked items concentrate in a low-dimensional subspace.
The research file's second gap names the resulting tension: should a safe pseudo-label have low or high entropy? Our claim resolves it by splitting the question in two. Low uncertainty over the final answer is desirable, but only if it survives meaning-preserving rewording. Low uncertainty over the path is a warning sign. Dealing with Synthetic Data Contamination in Online Continual Learning handles a similar problem in vision by keeping high-entropy images, because synthetic images draw overconfident predictions. That rule would discard confidently correct genuine solutions in reasoning tasks. The two-axis split avoids this because it never rewards uncertainty about the answer itself.
Research Program
The program has three parts.
The first part is a controlled contamination testbed. We start from an open model the file already uses for contamination studies, Qwen2.5-7B-Instruct (RL post-training detection). We create two contaminated variants from it. One is contaminated by supervised fine-tuning on a chosen subset of GSM8K and MSCalc test items, following the leakage protocol of A Narrowing Geometry. The other is contaminated by reinforcement learning with verifiable rewards on a subset of the RL-MIA items: AIME 2024/2025, Knights & Knaves, and SAT.
We then build an unlabeled self-training pool from three kinds of problem. The first is leaked items, partly paraphrased so that surface matching misses them. The second is unleaked items drawn from the same benchmarks. The third is freshly generated problems from DyVal, which builds reasoning tasks from randomly generated graphs and therefore cannot have been seen in training. Ground truth for every pool item is withheld from the training pipeline but kept for auditing. Because we know which items leaked, we can measure exactly what each filter selects.
The second part is the diagnostic. For a fixed self-training recipe, we measure how often each filter selects an item, conditioned on whether the model's answer is correct. The recipe draws $k$ samples per problem, takes the majority answer, keeps items whose vote share exceeds a threshold, and then fine-tunes on the kept items. The central quantity is the enrichment ratio: the selection rate for leaked items divided by the selection rate for unleaked items, restricted to items the model answers correctly. Comparing only correct items rules out the trivial explanation that leaked items are simply answered correctly more often. An enrichment ratio well above one on correct items would confirm that the confidence filter favors memorization. A ratio near one would mean contamination is not a selection problem in self-training, and the method below would be unnecessary. The diagnostic is useful whichever result appears.
The third part is the filter. For each problem we already have $k$ majority-vote samples. From these we compute path diversity as the mean pairwise normalized edit distance between reasoning traces. This is the statistic behind Min-$k$NN Distance, but here it comes for free from samples already drawn. We then generate $m$ meaning-preserving variants of the problem, with $m$ between 2 and 4, using the three perturbation types of CAP: reordering options or clauses, changing distractor details that do not affect the answer, and paraphrasing. We sample each variant a few times. Answer invariance is the fraction of variants whose majority answer matches the original.
A pseudo-label is kept when vote share is high, answer invariance is high, and path diversity is above a per-task floor. The floor is calibrated on a small labeled clean set. This filter plays the role UPS assigns to dropout variance, adapted to a generative model that is typically queried by sampling rather than by dropout.
We also run a white-box version as an analysis, not as the main method. It replaces edit distance with the eigenspectrum concentration index from A Narrowing Geometry, which measures how concentrated a model's internal activations are. This tests whether the black-box path signal tracks the internal subspace collapse. That link is the third gap in the research file.
Baselines
The naive reference is self-training with no filter: the model fine-tunes on the majority answer for every pool item. This shows how much any filter helps at all, and how much contamination the pool injects when nothing is screened.
The default competitor is the majority-vote confidence threshold, which is the filter actually used in practice. It plays the role that maximum-softmax thresholding plays in vision self-training. UPS reports that this kind of plain confidence-based pseudo-labeling reaches 22.02% error on CIFAR-10 with 1,000 labels, against 8.18% with uncertainty-aware selection (UPS). That result is a vision analogue, not an LLM number. A token-entropy threshold is run alongside it as a second confidence variant.
The strongest competitor is a two-stage pipeline. A dedicated contamination detector first removes suspected leaked items, and the majority-vote threshold is then applied to the remainder. The detector is either Min-$k$NN Distance, with a reported average AUC of 0.70 across 1.5B to 32B models (Structural Convergence), or Self-Critique, with a reported average AUC of 0.70 on Qwen2.5-7B-Instruct (RL post-training detection). This is the strongest competitor because it is exactly what a careful practitioner would assemble from existing tools. We also include a perplexity and Min-K% prefilter, even though its reported AUC on RL contamination is 0.42–0.53 (RL post-training detection). It is included to confirm that failure inside a training loop. An oracle that removes exactly the known leaked items sets the upper bound.
Fairness rules apply to all methods. Every method gets the same pool, the same base model, the same self-training recipe and number of fine-tuning steps, and the same total budget of generated tokens per pool item. When Min-$k$NN is given its native 32 samples, our filter is given the same budget, split between original and reworded prompts. Each threshold is tuned on the same small labeled clean set, with the same number of tuning trials.
Ablations isolate the key ingredient by running answer invariance alone (CAP-style perturbation used as a filter) and path diversity alone. If either single signal matches the combined filter, the two-axis claim fails.
No reported result was found on Lacuna for any method applied to pseudo-label selection in LLM self-training on a contaminated pool. All of these comparisons are therefore new measurements.
Our filter counts as beating the two-stage pipeline only if it meets two conditions at matched compute. First, it must select fewer leaked items. Second, it must yield equal or higher accuracy on fresh DyVal problems and on the contamination-proof SAT tier of P⁴Bench after self-training.
Why not just…?
Why not just keep the majority-vote confidence filter, since leaked answers are usually correct anyway? This alternative deserves a serious answer. If leaked items carry correct labels, training on them might be harmless. The diagnostic tests that directly. We compare self-training on a pool that includes the leaked items with self-training on the oracle-cleaned pool, and measure accuracy on fresh DyVal and P⁴Bench problems. If the two match on fresh problems, the only harm is inflated benchmark scores, and we will report the contribution as an evaluation-integrity result rather than a generalization result. Our expectation of real harm rests on two observations. Leaked items take up the selection budget and push out harder clean problems. Fine-tuning on collapsed paths may also narrow the model's reasoning further, the effect A Narrowing Geometry describes.
Why not just decontaminate the pool with a detector or n-gram matching first, then pseudo-label as usual? This is the main threat, and it is the strongest baseline above. N-gram matching needs the benchmark text and fails on paraphrase. Min-$k$NN, by contrast, was barely affected by paraphrased prompts, with AUC moving only from 0.72 to 0.71 (Structural Convergence). The case for our filter is that a contamination prefilter addresses only leaked items. It still passes clean items that the model answers confidently but wrongly, and that error is the original motivation for uncertainty-aware pseudo-labeling. Answer invariance under rewording targets both failures with one signal. The matched-compute comparison settles the question. If the two-stage pipeline equals our filter on both leaked-item selection and fresh accuracy, we will report that existing detectors suffice once they are placed in the loop. In that case the diagnostic becomes the main finding.
Why not just filter by perplexity, which is cheap? Perplexity rewards the same low-surprise behavior that memorization produces. Under reinforcement-learning contamination it is no better than chance, at 0.42–0.53 AUC (RL post-training detection). We include it so that this failure is measured inside self-training rather than assumed.
Proposed Experiments
The decisive first experiment is the enrichment ratio of the majority-vote filter on correctly answered items, measured separately for the fine-tuning-contaminated model and the reinforcement-learning-contaminated model. The prediction is that the ratio exceeds one more strongly under reinforcement-learning contamination, where path collapse is reported to be most severe.
The second experiment is the full self-training comparison against every baseline. It reports four quantities: precision of the selected pseudo-labels, the fraction of selected items that were leaked, accuracy on fresh DyVal tasks across difficulty levels D1–D4, and P⁴Bench SAT accuracy. It also reports the gap between post-training accuracy on the leaked benchmark and on fresh problems, which serves as a direct measure of score inflation.
The third experiment tests the white-box link. On the leaked GSM8K and MSCalc items, we check whether low path diversity tracks a high eigenspectrum concentration index from A Narrowing Geometry.
Finally, we test inheritance through distillation, which Structural Convergence reports as an AUC of 0.76 when auditing a distilled student. We distill a student from each contaminated teacher and repeat the diagnostic. The question is whether a student self-training on its teacher's leaked problems shows the same enrichment, and whether the filter removes it.
References
- In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
- Tri-net for Semi-Supervised Deep Learning
- USDNL: Uncertainty-Based Single Dropout in Noisy Label Learning
- Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
- Detecting RLVR Training Data via Structural Convergence of Reasoning
- A Narrowing Geometry in Contaminated Reasoning
- Dealing with Synthetic Data Contamination in Online Continual Learning
- DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
- CAP: Data Contamination Detection via Consistency Amplification
- $\text{P}^4$Bench: Contamination-Proof, Publicly Verifiable, and Privacy-Preserving LLM Evaluation via Zero-Knowledge Proofs