When a large model helps a smaller one learn, by showing it hints or worked steps during training, nobody decides carefully when that help should stop. Teachers of people have a name for doing this well: the gradual release of responsibility, which means handing over the work step by step so the learner is neither swamped nor dependent. We propose to make that hand-over decision from a split of the student's uncertainty into two parts. The first is uncertainty the teacher could resolve, because the student lacks a piece of knowledge. The second is uncertainty that persists whatever the teacher says, because the student's reasoning is simply noisy. The policy that follows is concrete. Give help only where it would move the student. Give the least specific help that does so. Stop helping, and do not add more context, where the student's errors are noise rather than missing knowledge.
The question matters because both failure modes already appear in the file. Heavy, specific help from a language-model tutor goes wrong exactly where the learner is nearly right. On almost-correct student code, the highest rungs of a feedback ladder (exact bug location, direct edit) invent bugs that are not there (Generating Feedback-Ladders for Logical Errors in Programming using Large Language Models). Pouring more peer context into a model can also make its reasoning less stable rather than more correct (Epistemic Gain, Aleatoric Cost). A rule for when to help, and how much, should know which of these situations it is in.
Core Claim
The quantity that a coaching system needs, the "zone of proximal development," is in effect a measure of epistemic uncertainty relative to the teacher. The zone of proximal development is the set of things a learner can do with help but not yet alone. Shared Autonomy for Proximal Teaching estimates it as the gap between assisted and unassisted performance on driving sub-skills, using offline lap data and coach labels. We claim the same gap can be estimated per problem, before grading and without an answer key, as the information a teacher hint carries about the student's answer.
Let $Y$ be the student's final answer sampled on a problem and $Z$ a hint drawn from the teacher. Then
$$H(Y) = I(Y;Z) + H(Y \mid Z),$$
where $H(Y)$ is the entropy of the student's answer distribution, $I(Y;Z)$ is the mutual information between answer and hint, and $H(Y \mid Z)$ is the entropy that remains once a hint is given. We call $I(Y;Z)$ teachable uncertainty: it is high exactly when help changes what the student does, which is the zone of proximal development. We call $H(Y \mid Z)$ residual noise: help does not remove it, and by UMAD's evidence extra context may enlarge it.
On competition math, residual noise is not aleatoric uncertainty in the classical sense, since each problem has one right answer. It is reasoning instability. We say so plainly and test whether separating it out matters at all.
Research Program
The setting reuses UMAD's heterogeneous pair: Qwen2.5-3B as student and Qwen3-4B as teacher. That paper showed the stronger model acting as an anchor that lifts the weaker one during debate. The teacher can offer help at four rungs, adapted from the feedback ladder of Heickal and Lan: a verdict on whether the student's current answer is right, a conceptual hint about the next idea, one worked step, or a full solution.
The student is trained with reinforcement learning against verifiable answers, using Group Relative Policy Optimization as UMAD does. Each training problem is posed with whatever help a gate decides.
The gate works in two stages.
- Cheap screen. It first computes the student's own epistemic signal with no teacher call, using the iterative self-prompting mutual-information test of To Believe or Not to Believe Your LLM. That paper feeds a model's earlier answers back into its context and reads instability across steps as ignorance. Problems where the student is stable and its total answer entropy is low are released: no help is given.
- Teachability estimate. For the remaining problems, the gate estimates $I(Y;Z)$ by sampling student answers with and without the lowest rung and clustering final answers by value. This is the suggestibility idea of the in-context learning test in Distinguishing the Knowable from the Unknowable with Language Models, which detects missing knowledge by how sharply entropy collapses when a candidate is placed in context. That paper reports AUROC of about 0.7 for this unsupervised test and above 0.9 for supervised middle-layer probes. We use the test, not as a hallucination detector, but as a measure of whether help will work.
The gate then acts on the estimate. If teachable uncertainty is high, it climbs the ladder only until the student's answer entropy collapses. If residual noise dominates, it gives a verdict only and asks for a fresh attempt, adding no further context. As training proceeds and teachable uncertainty on a cluster of related problems falls, that cluster's help is withdrawn. Withdrawal is how release happens.
The gate itself has no learned parameters beyond two thresholds. These are calibrated on a held-out split, following the residual-based, validation-calibrated weighting of the two uncertainty components in CLEAR. CLEAR's own guidance is that roughly 150 validation points suffice.
This design differs from each nearby system in a specific way:
- UMAD decomposes uncertainty inside a fixed-round debate and uses it to shape rewards. It never withdraws the stronger agent. Its reported accuracy is the weak agent's accuracy while still helped.
- Situational Confidence Assistance for Lifelong Shared Autonomy (CASA) hands control back and forth between a human and a robot arm using one scalar confidence. It therefore cannot tell a jittery operator from a novel goal.
- Z-COACH fades help per sub-skill, but only from offline performance gaps and human coach text.
- The feedback-ladder work picks rungs statically.
None of these uses a split of uncertainty to decide both how much help to give and when to stop. None measures the outcome that gradual release actually cares about, which is how well the learner performs once help is gone.
Baselines
The naive reference point is the student trained by the same reinforcement-learning recipe with no teacher at all. It shows how much any scaffolding buys.
The opposite pole is always-on help: every training problem comes with a teacher hint. This measures learned dependency, since such a student may score well with help and poorly without it.
A fixed fading schedule withdraws help from all problems after a set fraction of training steps. It is the time-based rule a skeptic would propose.
The strongest existing method on this question is UMAD itself, run in its heterogeneous configuration. UMAD reports that by the fifth debate round the weaker Qwen2.5-3B agent's MATH500 accuracy "rose dramatically from 54.8% to 66.5%." That figure is accuracy with the teacher in the loop. We found no reported result on Lacuna for the solo accuracy of a UMAD-trained weak agent with the stronger agent removed.
We will reproduce UMAD on the same training problems, with the same base models, the same reinforcement-learning steps and the same hyperparameter search budget. We will then evaluate its student both assisted and alone. Beating UMAD means one of two things:
- higher solo student accuracy on MATH500, AMC23 and AIME24/25 with no more teacher tokens spent during training, or
- equal assisted accuracy at a substantially lower teacher-token cost.
Two further baselines isolate the key ingredient. The first is a gate that thresholds total answer entropy $H(Y)$ alone. The second is a gate using only teachable uncertainty $I(Y;Z)$, without the residual-noise branch. Comparing them separates the value of measuring teachability from the value of splitting off noise.
Finally, an oracle gate uses the answer key to compute the actual assisted-minus-unassisted accuracy per problem, which is the Z-COACH quantity. It gives an upper reference for how good any label-free estimate could be.
Why not just…?
Why not just gate help on total entropy? Total entropy treats a student who is torn between two answers because it lacks a fact the same as one whose sampled chains wander randomly. Only the first benefits from a hint. On questions with several valid answers, To Believe or Not to Believe Your LLM found that total-entropy-style signals flag legitimate diversity, while mutual information does not. That evidence concerns ambiguity rather than reasoning noise, so it does not settle our case. The total-entropy ablation will: if it matches the decomposed gate on solo accuracy and teacher cost, the split adds nothing here and we will report that.
Why not just fade help when the student starts getting problems right? This is the real threat. With verifiable answers during training, accuracy-based fading per problem cluster is cheap, and the oracle zone-of-proximal-development gate is its idealized form. Correctness alone cannot say whether a wrong student would be helped by a hint or merely needs another attempt. The decomposed gate can. Its other claimed advantage is that it needs no answer key, which matters for unlabeled or open-ended problems.
We will run accuracy-based fading as its own arm. If it equals the decomposed gate on labeled math, the contribution narrows to label-free settings. We will then test that narrower claim by training on a pool where half the problems have their answers hidden from the gate.
Why not just distill the teacher's solutions into the student? Supervised fine-tuning on teacher worked solutions is a strong, simple way to raise solo accuracy. It needs no gate. We will include it with teacher tokens matched to the gated arm. The prediction from the gradual-release view is that distillation helps most on easy problems and adds instability on problems the student was already nearly solving. This mirrors the specificity-induced errors Heickal and Lan saw on almost-correct code. If distillation simply wins at matched cost, the gating idea is not worth its complexity for this pair of models.
Proposed Experiments
Training uses MATH training problems. Evaluation uses MATH500, GSM8K, AMC23 and AIME24/25, the suite UMAD reports on. The primary metric is the student's accuracy with no teacher present. Secondary metrics are teacher tokens consumed during training, accuracy with help at test time, and the gap between the two, which is our measure of dependency.
The first decisive check is whether teachable uncertainty predicts per-problem help benefit. On held-out problems, we will compute the AUROC and rank correlation of $I(Y;Z)$, of total entropy, and of the cheap self-prompting score against the oracle assisted-minus-unassisted gain. If $I(Y;Z)$ does not beat total entropy here, the central claim fails before any training is done.
The second check is whether the residual-noise branch is right that more context hurts noisy problems. On problems flagged as noise-dominated, we will compare student accuracy under verdict-only help against full-solution help. The prediction is that full solutions give little or negative gain there, consistent with UMAD's account of how peer context destabilizes reasoning.
The third check is the full training comparison across all arms at matched compute, with three seeds each. Solo accuracy is reported with confidence intervals, and teacher-token cost is plotted against solo accuracy so that cheaper and better are visible together.
A small transfer probe then asks whether the same gate, unchanged, selects feedback rungs sensibly on the 2nd CSEDM Data Challenge programming data. The test is whether it avoids the location and edit rungs on high-scoring submissions, where Heickal and Lan documented invented bugs. That would indicate the policy is not specific to math.
References
- Generating Feedback-Ladders for Logical Errors in Programming using Large Language Models
- Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning
- Shared Autonomy for Proximal Teaching
- To Believe or Not to Believe Your LLM: Iterative Prompting for Estimating Epistemic Uncertainty
- Distinguishing the Knowable from the Unknowable with Language Models
- CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk
- Situational Confidence Assistance for Lifelong Shared Autonomy