When a language model solves a hard math problem, the most reliable cheap trick is to sample many step-by-step solutions and take a majority vote over their final answers. The vote treats every sampled solution as an independent witness. It is not one. A model that has been heavily specialized on math, by distillation or reinforcement learning, tends to produce many near-copies of the same derivation, so ten votes for a wrong answer may really be one flawed argument repeated ten times. We propose to score each candidate answer by how many structurally different derivations reach it, using the redundancy penalty from maximal marginal relevance. Maximal marginal relevance is a classic summarization rule that picks the next sentence by its relevance minus its similarity to sentences already picked. When independent support for the leading answer is thin, the system samples more solutions while explicitly steering away from derivations it has already seen. It never asks the model to critique and rewrite its own answer.
The second half matters because of a firm negative result: models asked to review and revise their own reasoning, with no outside checker, more often talk themselves out of right answers than into them. Our design keeps the useful part of self-correction, which is noticing that the current evidence is weak and spending more effort, and drops the harmful part, which is editing a committed answer. The claim to test is concrete. On specialized open models, wrong majority answers are disproportionately supported by structurally redundant chains, so discounting that redundancy should beat plain majority voting at the same generation budget, and the gain should be largest exactly where voting fails.
Core Claim and Positioning
Three findings motivate the proposal. Large Language Models Cannot Self-Correct Reasoning Yet shows that prompting a model to self-critique degrades accuracy on GSM8K, the grade-school math word-problem benchmark (GPT-4 from 95.5% to 89.0%; Llama-2-70b from 62.0% to 36.5%). The same paper shows that majority voting over sampled reasoning chains, known as self-consistency, beats multi-agent debate at a matched number of responses (88.2% versus 83.0% with nine responses). Can Large Language Models Really Improve by Self-critiquing Their Own Plans? traces the failure to verification: a GPT-4 verifier accepts 84.45% of incorrect Blocksworld plans. Together these papers constrain the design. Any loop that lets the model overwrite answers on the strength of its own verdicts is suspect, and self-consistency is the baseline to beat. ChainPrune supplies the third observation: sampled long reasoning chains carry heavy structural redundancy, with semantically equivalent steps recurring across paths. ChainPrune uses this offline, merging equivalent steps into a tree and fine-tuning toward short paths. It never asks what redundancy does to the evidential weight of samples at inference time.
The method comes from the maximal marginal relevance literature, which formalizes greedy selection under a redundancy penalty (Manifold-Ranking Based Topic-Focused Multi-Document Summarization shows that relevance-only ranking gives redundant items identical high scores). That literature names an open problem that is central here: surface similarity cannot separate repeating a fact from independently corroborating it. In reasoning this distinction is the whole point. Two chains that reach 42 through the same intermediate quantities are one piece of evidence. Two chains that reach 42 by an algebraic route and a counting route are two.
The closest prior work differs as follows. Training Verifiers to Solve Math Word Problems samples 100 chains and re-ranks them with a trained verifier, but treats the samples as exchangeable. DICE trains a small model to correct one chain produced by a larger model. Beyond Output Critique refines a single attempt through an extracted task template. None of them reweights evidence across multiple chains by how independent those chains are.
Research Program
The core quantity is the independent support of an answer. Take the $n_a$ sampled chains whose final answer is $a$. Order them greedily by maximal marginal relevance: at each step, pick the chain that maximizes $\lambda$ times its relevance (the model's length-normalized log-probability, or a verifier score when one is available) minus $(1-\lambda)$ times its maximum similarity to chains already picked. Then define
$$S(a)=1+\sum_{k=2}^{n_a}\Big(1-\max_{j<k}\,\mathrm{sim}\big(c_{(k)},c_{(j)}\big)\Big),$$
where $c_{(k)}$ is the $k$-th chain in that order and $\mathrm{sim}\in[0,1]$ is a structural similarity. $S(a)$ equals $n_a$ when all chains are fully distinct and approaches 1 when they are copies. The prediction is the answer with the largest $S(a)$. As an order-free ablation, we also compute a kernel effective count in which each chain contributes the inverse of its summed similarity to all chains sharing its answer.
Everything depends on the choice of $\mathrm{sim}$. We will compare three options. The first is sentence-embedding cosine over steps, the classic choice. The second is Jaccard overlap of the intermediate commitments extracted from each chain: the numeric values, equations, and named sub-results for math, and the reached subgoal states for Blocksworld. The third is a learned similarity, trained on pairs of chains labeled as equivalent derivations using ChainPrune-style step merging as weak supervision. Our hypothesis is that intermediate-commitment overlap captures "same argument" far better than embeddings, which conflate shared vocabulary with shared logic.
The second component is adaptive, diversity-steered resampling. This is the replacement for self-correction. After an initial batch, if the margin between the top two values of $S$ falls below a threshold tuned on held-out data, we draw further chains. Each new chain is accepted only if its maximum similarity to existing chains is below a cutoff. If that rejection rate gets too high, the prompt instead lists the intermediate commitments already used and asks for a solution that avoids them. Existing chains are never edited. This means the specific mechanism that produces correct-to-incorrect flips in the self-correction literature is absent by construction. New chains can still shift the vote, and we measure flip rates directly.
The application regime is domain-specialized small models. Our primary model is AceReason-Nemotron-14B, which is trained with reinforcement learning against math and code checkers. We compare it with its distilled pre-RL starting point and a generalist of similar size. We expect redundancy to be worst, and the method most useful, after reinforcement learning sharpens the output distribution. This is a prediction about specialization, and the experiment can confirm or refute it. Where an external checker exists we use it only to score results, never inside the method, because we target settings where no such oracle is available at inference.
Baselines
The naive reference point is a single greedy-decoded chain per problem. This number anchors how much any multi-sample method adds.
The strongest training-free competitor is self-consistency, majority voting over $N$ temperature-sampled chains. Large Language Models Cannot Self-Correct Reasoning Yet reports 88.2% on GSM8K with nine responses, beating debate at 83.0%. It is the strongest competitor because it already captures most of the gains of iterative schemes at equal cost. We run it with the identical model, prompt, temperature sweep, and total generated-token budget as our method. That includes the tokens spent on adaptive resampling: self-consistency is granted the same average number of chains per problem. On AIME with AceReason-Nemotron-14B, the file reports only avg@64 accuracy (67.4% on AIME 2025, versus 50.2% for the distilled base), not a majority-vote figure. We found no reported result on Lacuna for self-consistency of this model on AIME, so we will measure it ourselves. Beating it means a higher answer accuracy at a matched token budget, with the confidence interval excluding zero across seeds and problem resampling.
The strongest trained competitor is verifier-weighted selection in the style of Training Verifiers to Solve Math Word Problems, where a 6B model with a verifier reached about 35% on GSM8K against about 33% for a fine-tuned 175B model. Our verifier is a LoRA (low-rank adapter fine-tuning) head trained on 1,000 NuminaMath examples, following the recipe in Can A Base LLM Verify Its Own Math?, which elicited explicit verdicts from Llama-3.1-8B. We run both best-of-$N$ and verifier-score-weighted voting. The same verifier score then serves as the relevance term in our ordering, so the comparison isolates the redundancy discount. Beating this baseline means our method plus the verifier outperforms verifier-weighted voting with the same verifier and the same samples.
Three ablations isolate the key ingredient. The first is redundancy discounting without adaptive resampling, applied to a fixed batch. The second is adaptive resampling triggered by the vote margin but with plain counting. The third swaps the structural similarity for embedding cosine. If the gain survives only with structural similarity, the claim about repetition versus corroboration is supported. A final control is exact-duplicate removal (discarding chains whose extracted commitments are identical), which tests whether a simple dedupe gets most of the effect.
Why not just…?
Why not just draw more samples for self-consistency? If wrong majorities come from many near-copies of one flawed argument, extra samples from the same distribution mostly add more copies. Accuracy then plateaus while the wrong answer's vote share stays stable. The hard-set result in Can A Base LLM Verify Its Own Math?, where a small model has pass@1024 of zero, shows that raw sample count cannot help when the right answer is never generated. That case is outside any voting method's reach, including ours, and we will report it separately. The real test is the scaling curve: we plot accuracy against tokens for both methods up to 64 chains. If plain self-consistency closes the gap as $N$ grows, our method is only a sample-efficiency gain, and we will say so.
Why not just raise the temperature or vary the prompt to get diverse samples? This is a serious threat. Diversity at generation time might remove the redundancy that our scoring discounts. Higher temperature also adds noisy, incorrect chains, and voting depends on the correct answer being the mode. We include temperature and prompt-variation sweeps under self-consistency. We report the structural redundancy of each sample pool, so the analysis can show whether gains track redundancy itself or merely how it was reduced.
Why not just use a trained verifier and pick the top-scored chain? Verifiers are imperfect in correlated ways. The 84.45% false-positive rate in Can Large Language Models Really Improve by Self-critiquing Their Own Plans? is the extreme case, and a verifier that accepts one flawed derivation will tend to accept all its copies. Our method uses the verifier as the relevance term, not as a substitute, so the comparison above directly measures whether discounting adds anything on top of the verifier.
Why not just use external checkers or offline pruning? Executable checkers, such as those used to train AceReason-Nemotron, and the VAL plan validator dominate where they exist, and we do not compete with them there. Our target is MedQA-zh and multi-hop question answering, where no checker exists. ChainPrune shortens chains through training but does not change how evidence across chains is weighed. Our method can be applied on top of a ChainPrune-trained model, and we will test that combination.
Proposed Experiments
The decisive first check is cheap and could kill the idea. On existing samples (64 chains per problem on GSM8K, MATH500, AIME 2024/2025, and Blocksworld), we compare the structural redundancy of chains backing wrong majority answers with that of chains backing correct ones. If wrong majorities are not more redundant, discounting cannot help and we stop. We also test whether the margin of $S$ predicts correctness better than vote share, measured as AUROC (the area under the ROC curve) for the correct-versus-wrong prediction. Repeating this analysis across the generalist, distilled, and reinforcement-learned models tests whether specialization worsens redundancy.
If the premise holds, the main experiments compare all baselines at matched token budgets on GSM8K, MATH500, AIME, Blocksworld (VAL used only for scoring), and MedQA-zh, the last as the setting with no checker. We report accuracy, accuracy on the subset where self-consistency's majority is wrong, and correct-to-incorrect flips introduced by resampling. Comparing these flip counts with the flips caused by intrinsic self-correction under the protocol of Large Language Models Cannot Self-Correct Reasoning Yet tests whether adding new evidence avoids the failure that revising old answers produces. We also report accuracy on BigBench Hard, where Specializing Smaller Language Models towards Multi-Step Reasoning found specialized models lose general reasoning. This checks whether redundancy discounting partly offsets that loss or simply inherits it.
References
- Large Language Models Cannot Self-Correct Reasoning Yet
- Can Large Language Models Really Improve by Self-critiquing Their Own Plans?
- ChainPrune: Evaluating and Reducing Redundancy in Long Chain-of-Thought Reasoning
- Manifold-Ranking Based Topic-Focused Multi-Document Summarization
- Training Verifiers to Solve Math Word Problems
- DICE: Structured Reasoning in LLMs through SLM-Guided Chain-of-Thought Correction
- Self-Correction via Task Distillation
- Can A Base LLM Verify Its Own Math? A Pre-Registered Discriminator-Generator Asymmetry Test On AIME And Four Corrective Probes
- AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
- Specializing Smaller Language Models towards Multi-Step Reasoning