Repairing Teacher Reasoning Traces at the First Broken Step

When a small "student" model is trained on step-by-step solutions written by a larger "teacher" model, the student can only be as good as the solutions it sees. Today's pipelines protect students in one of two ways. Some throw away any solution whose final answer or executed code is wrong. Others ask a language model to critique and rewrite solutions in plain language. Throwing solutions away means the student never sees the hardest problems, which are exactly the ones the teacher rarely gets fully right. Prompted rewriting introduces new, unverified mistakes. This proposal tests a third option. Rewrite each teacher solution so that every step can be checked mechanically. Use an external checker, such as a theorem prover or a computer-algebra system, to find the first step that fails. Replace that one step with a verified correction, and let the teacher continue from the repaired, already-verified prefix.

The bet is that most failed teacher traces are mostly right. They contain a sound plan with a single broken inference or arithmetic slip. A trace repaired at the exact point of failure is therefore a cheap way to get fully verified solutions to problems beyond the teacher's own success rate. If the bet holds, the ceiling a student inherits is set by teacher-plus-checker rather than by the teacher alone. If it fails, the failure is informative too: it would mean that broken traces are broken in their planning, not their details, and that resampling is the right use of compute.

Core Claim

Under a fixed teacher-generation budget, localized repair yields fully step-verified training traces for a meaningfully larger set of hard problems than rejection sampling. Hard problems here are those where the teacher's verified pass rate over $k$ samples is near zero. Localized repair means checker-identified first error, a verified patch, and teacher continuation from the verified prefix. Students fine-tuned on the resulting corpus should outperform students trained on equal-budget filtered corpora. The gain should be concentrated on the hard tail: deeper ProofWriter proofs, and problems where the teacher's pass rate is low.

A second, more specific claim follows from the same step-level checking. Once every step must cite its premises, the dependency graph of a verified trace shows which steps are actually used to reach the answer. Deleting unused steps gives a symbolically grounded way to shorten verbose teacher traces. Unlike prompted shortening, it cannot remove a lemma the proof needs.

How This Differs From Existing Work

The step format comes from Deductive Verification of Chain-of-Thought Reasoning. That work has the model number its premises and make each step cite the premises it uses, then has an LLM verify each step in isolation. It discards any chain with an invalid step. We reuse the citation format, but replace the LLM verifier with formal checkers and replace discarding with repair. That paper also reports that GPT-3.5-turbo verifying whole chains reached only 52% accuracy, which motivates a mechanical checker. And it observes that correct final answers often rest on flawed steps, which motivates step-level rather than answer-level filtering.

The repair mechanism borrows its logic from abductive correction on puzzles. ABL-Refl and NASR learn a mask that flags suspected erroneous outputs and send only those to a solver; ABL-Refl reaches 97.4% on symbolic Sudoku. Both stay within grids, graphs, and visual scene relations at inference time. Our setting is different in a way that simplifies the method. For step-structured text, the checker itself locates the first failing step, so no mask needs to be learned. The open question shifts from "where is the error" to "does a spliced, repaired trace teach a student as well as a clean one."

Three pipelines filter synthetic reasoning data by rejection: Caco (Python execution plus agreement between the code and the language answer), TableLLM, and constraint-based SMT data generation. We keep their verification philosophy and change what happens on failure.

Two pipelines already revise failing outputs using solver or execution feedback. Logic-LM feeds solver error messages back to the LLM at inference time. CodeI/O++ re-executes predicted inputs and supplies the outcome as a second revision turn. Both regenerate the whole output from coarse feedback and never repair intermediate rationale steps. Whole-trace revision with feedback is therefore our most important baseline, not a straw man.

On the distillation side, two methods target the teacher bottleneck without any grounding. Local Naturalness selects teacher traces by how probable the student finds each local window of steps. ORION has the student rewrite teacher traces after examining its own mistakes, producing traces 30–40% shorter. Neither checks whether any step is valid, and ORION's own limitation is that the self-analysis can hallucinate.

Research Program

The pipeline has four stages.

The first stage is step-structured generation. An open teacher in the 70B class, of the kind used as a teacher in the Local Naturalness work, writes solutions in the citation format. Each step names its parent premises or steps.

The second stage is per-step formalization and checking. For logic problems, each step plus its cited premises is translated into first-order logic, and Prover9 or Z3 checks whether the cited premises entail the step. For arithmetic and algebra, each step's equation is checked with a computer-algebra system against the values established by its cited steps.

Translating a single short step is meant to be less fragile than translating a whole problem, which is where LINC and FOLIO report that small semantic errors make the prover fail. FOLIO puts natural-language-to-logic syntactic validity at roughly 93%, but syntactic validity says nothing about whether the meaning was translated correctly. We will measure that fidelity directly against FOLIO's expert logic annotations. Steps that cannot be formalized are labeled "unchecked" rather than "passed." Traces are reported by their checked fraction, so the method cannot quietly launder unverified content.

The third stage is localized repair. At the first failing step, the system proposes a replacement in one of two ways. For arithmetic, it recomputes the value. For logic, it asks the teacher for candidate conclusions from the same cited premises and keeps one the prover accepts. The teacher then continues from the verified prefix. The loop repeats up to a small fixed number of repairs. Only traces whose answer matches the gold answer and whose checked steps all pass enter the corpus.

The fourth stage is dependency pruning. Steps that the final answer does not depend on are removed, producing a shortened variant of each trace.

Students are Qwen2.5-7B and LLaMA-3.1-8B, both used as students in the Local Naturalness work. They are fine-tuned on each corpus with identical hyperparameters and token counts.

Baselines

The naive reference point is supervised fine-tuning on raw teacher traces with no filtering. This tells us how much any filtering matters in this setting.

The standard practice is outcome-only rejection sampling: keep traces whose final answer is correct, which is the core of Caco-style filtering. It is a strong competitor because it is cheap and, at large scale, has produced strong students. Caco reports 92.6% on GSM8K and 82.4% on MATH for Qwen2.5-7B trained on Caco-1.3M. That corpus is far larger and more code-anchored than ours, so we quote those numbers as context rather than as a target. We will run outcome-only rejection ourselves, on our problems, with our teacher.

Step-verified rejection uses the same formal checkers but discards any trace with a failing step. It is the symbolic analogue of the voting scheme in Deductive Verification. It is the critical ablation: it holds verification fixed and removes only the repair.

The strongest existing method for our question is whole-trace regeneration with checker feedback. When a trace fails, the teacher receives the checker's message and rewrites the entire solution, in the manner of Logic-LM's self-refinement loop and CodeI/O++. A second ablation separates the value of symbolic localization from the value of repair: an LLM critic, rather than the checker, picks the step to repair.

Finally, we run Local Naturalness selection and ORION-style student rewriting on the same problem pool. These represent the ungrounded responses to the teacher bottleneck.

All corpora come from the same problem pool and teacher. They are matched on total teacher tokens generated, including all resamples, regenerations, and continuations. Fine-tuning compute is fixed per student, and every method gets the same tuning budget.

Beating the strongest baseline means a statistically reliable gain over whole-trace regeneration on the hard-tail splits, at equal teacher tokens. No reported result was found in the research file for any of these baselines on student distillation in exactly this setting. Logic-LM's 78.1% on FOLIO with GPT-4 plus a solver is an inference-time system, not a distilled student. It serves as an upper reference, alongside FOLIO's 65.9% for a fine-tuned Flan-T5-Large.

Why not just…?

Why not just resample the teacher more and reject failures? This is the real threat. If failed traces are wrong in their plans rather than their details, resampling at equal budget will match repair. The decisive measurement is coverage. For each problem we compare the fraction with at least one fully verified trace under resampling versus under repair, at matched teacher tokens, binned by the teacher's verified pass rate. If repair does not raise coverage in the lowest bins, the core claim is false and we will say so.

Why not just feed the checker's error back and let the teacher rewrite everything? Logic-LM and CodeI/O++ show that feedback-driven revision helps. But regenerating a whole trace gives every previously correct step a fresh chance to break. We test this directly with the whole-trace regeneration baseline. We also measure how often regenerated traces introduce new failing steps compared with continuations from a verified prefix.

Why not just ask the model to find and fix its own mistakes? The research file's evidence is against ungrounded self-checking. Whole-chain LLM verification reached 52% in the Deductive Verification study. Base language models showed 19–43% constraint violations until a MaxSAT solver was applied in ParRoT-Con. Synthetic Error Injection Fails to Elicit Self-Correction In Language Models reports that self-correction tends to fail without external signals. The ORION and LLM-critic baselines quantify this in our setting.

Why not just filter by student naturalness? Local Naturalness reports large gains, for example GPQA-Diamond rising from 0.611 to 0.702 relative to global selection. But it scores how familiar a step looks to the student, not whether the step is valid, so it can keep plausible fallacies. We will report the checker-measured invalid-step rate of the traces it selects. We will also test whether combining it with our repaired corpus adds to the gain, since the two are not mutually exclusive.

Proposed Experiments

The logic suite is ProofWriter, split by proof depth, plus FOLIO, LogicalDeduction, and AR-LSAT, with training problems drawn from their training sets. The math suite is GSM8K and MATH-500. MATH is included specifically to see where step formalization breaks down.

The metrics are:

  • student accuracy;
  • accuracy on the hard tail, defined by teacher pass rate and by ProofWriter depth of 4 or more;
  • verified coverage per teacher token;
  • the invalid-step rate of the student's own generations, as judged by the same checker;
  • the trace length of the pruned variants.

Four checks decide the outcome:

  1. Repair must beat both step-verified rejection and whole-trace regeneration on hard-tail coverage and student accuracy at matched budget.
  2. Gains must concentrate where predicted: low-pass-rate problems and deep proofs, not easy problems.
  3. A human audit of a few hundred repaired traces must show that the checker's "pass" corresponds to semantically correct steps, with the measured rate of passes caused by mistranslation reported.
  4. On deep ProofWriter splits, students trained on dependency-pruned traces should match unpruned ones, while ORION-style shortened traces lose accuracy as depth grows. This would resolve whether shortening teacher traces strips out necessary intermediate lemmas.

A negative result on check 1, combined with a positive result on check 4, would still leave a useful and narrower contribution: symbolic pruning of verbose teacher traces that preserves the steps a proof needs.