A Separate Agent for Withdrawing Help in Multi-Agent Tutors

Students who have spent years in lecture halls often resist being asked to think for themselves. Researchers on teaching call this pedagogical inertia: when a course shifts to self-paced or inquiry-based formats, many students lack the habits of self-regulation the format assumes. They stall, ask to be told, or disengage. Large language model (LLM) tutors face the same tension in miniature. If the tutor tells the answer, the student learns to wait for it. If it withholds help too long, the student gives up.

We propose a tutor in which one specialized agent decides only how much help to give on each turn. Other agents decide what to say within that limit. The help-deciding agent sees a compact record of the learner's history across many problems, not the raw conversation. We will test it against simulated students whose dependence on help actually persists from problem to problem, and then against a short classroom study. The claim to test is whether separating "how much help" from "what words" lets a tutor reduce help over weeks without triggering collapse. A simple counting rule might do just as well, and we compare against one directly.

Core Claim

Current multi-agent tutors split tutoring into roles such as hinting, correcting misconceptions, motivating, and prompting reflection. The work closest to ours does this and then has the role agents vote on each utterance (Voting Protocols as Coordination Mechanisms for Role-Constrained Multi-Agent Tutoring Systems). It finds that the voting rule quietly acts as a teaching policy. Its interactions average about 2.4 turns with LLM-simulated students, and the voting rule never changes during an interaction. The amount of help a student receives therefore emerges from a per-turn vote that has no memory of whether this student has been getting more dependent over the past twenty problems.

Our claim is that the help level should be a separate decision, made by a dedicated agent with cross-session memory and a deliberately narrow view of the dialogue. We argue this decision suffers from two pressures when it is entangled with text generation:

We hypothesize that a help-level decision made in isolated, compact context will reduce help faster with fewer breakdowns. The comparison is against the same decision made inside a single prompt, and against the decision made by voting.

The second half of the claim concerns the evaluation itself, and it is what makes the first half testable. Existing tutoring evaluations rely on LLM student personas:

A prompted persona has no persistent dependence. Its behavior can look stubborn only because the underlying LLM repeats itself, which is the machine kind of inertia, not the human kind. We therefore build simulated students whose dependence and frustration are explicit state variables, updated by code rather than by the LLM. We ask whether tutor rankings change when such state exists.

Research Program

The tutor

The tutor reuses the four role agents of the voting-protocol work as proposal generators: scaffolding, misconception, motivation, and metacognition. Before any proposal reaches the learner, an autonomy controller sets a help budget on a ladder grounded in existing taxonomies. The ladder runs from probing, through focusing attention on the error and hinting, to giving a worked step, and finally telling. It maps onto MathDial's Probing, Focusing, and Telling moves and onto the eleven finer intents of Intent Matters: Enhancing AI Tutoring with Fine-Grained Pedagogical Intent Annotation. Proposals above the budget are discarded or rewritten. The remaining candidates are chosen by ranked voting.

The controller's input is a short structured record. It contains:

  • whether the student's latest attempt differs from the previous one, which separates productive from repetitive struggle;
  • turns since the last progress;
  • affect markers extracted by a separate classifier call;
  • the help level previously needed on problems sharing the same skills.

The last item is what lets the controller lower help across sessions. It is also what no current system tracks at the dialogue level. GenMentor tracks engagement only at the scale of whole study modules. KELE (KELE: A Multi-Agent Framework for Structured Socratic Teaching with Large Language Models) fixes the order of Socratic stages through a planner-executor hierarchy. Our controller fixes no order and only bounds help.

The simulated students

Each simulated learner carries four kinds of state:

  • per-skill mastery;
  • dependence, which rises when the learner is told an answer without having made an attempt and falls after unassisted success;
  • frustration, which rises with unproductive turns and triggers simulated quitting past a threshold;
  • an initial dependence level representing a lecture-habituated background.

The update rules are written in code. An LLM only turns the current state into an utterance.

We do not have real student interaction logs to fit these rules, and the file contains none. We therefore treat the rules as explicit, varied assumptions rather than calibrated facts. We check them against qualitative patterns from the inertia literature:

We also run a diagnostic on standard persona simulators. We swap the tutor's history mid-trajectory and test whether the persona's next behavior tracks that history or merely its own previous utterance. This test asks whether prompted personas capture any persistent dependence at all.

Tasks and horizon

Tasks come from GSM8k problems as used in MathDial, and from the SciQ and HumanEval adaptations used in the voting-protocol study. Each simulated learner works through a sequence of 40 problems with overlapping skills, split into sessions. The sequence ends with an unassisted test on new problems. Session boundaries matter because Chen et al.'s novelty-decay result concerns how effects change over weeks, which single episodes cannot show.

Baselines

Do-nothing reference. A tutor that simply solves the problem and explains the solution: help is always at maximum. It sets the floor for unassisted test accuracy and the ceiling for short-term success. Every system must beat it on the final unassisted test, or the whole enterprise is moot.

Naive tutor. A single frontier model prompted to be a patient Socratic tutor that gradually reduces hints. MathDial reports 66% answer leakage for zero-shot ChatGPT, and 1% to 4% leakage plus an equitable-tutoring score of 2.19 versus 1.43 for its fine-tuned Flan-T5. We run the same model family as our agents with the same token budget. Prompt tuning gets the same number of development iterations we spend on the controller prompt.

Strongest existing multi-agent method. The four-agent committee with ranked voting from the voting-protocol study. It reports a +0.30 code-execution gain and +0.21 prose-explanation gain on HumanEval with ranked voting, and +0.10 learning gain on SciQ with approval voting, all relative to a single agent. We reimplement it with identical role prompts, base model, and per-turn token cap, including its fixed tie-break order. Beating it means:

  • higher unassisted test accuracy at sequence end;
  • a lower quit rate under high initial dependence;
  • no worse MathDial-style Success@k at matched Telling@k.

All three must hold with confidence intervals over simulated learners and seeds. A gain on one axis bought with a loss on another does not count.

Rule-based fading. The same multi-agent pipeline with the controller replaced by a classic contingent rule: raise the help level one step after a failed attempt and lower it one step after an unassisted success, with per-skill memory. No reported result found on Lacuna for this baseline in LLM tutoring. It is the most dangerous competitor, because it captures cross-session memory without any LLM judgment.

Ablations that isolate the key ingredient:

  • the controller's exact logic and record folded into the single prompt that generates the reply, which tests separation itself;
  • the controller given the full dialogue instead of the compact record, which tests whether conversational inertia is the operative harm;
  • the controller with cross-session memory removed.

Why not just…?

Why not just prompt one strong model to fade its help? The 66% leakage figure from MathDial says unguided models tell readily. A careful prompt may still do much better than that number suggests. The in-prompt ablation answers this directly: same logic, same information, one model rather than two. If it matches the separated controller, separation is not the mechanism. We would then report that the cross-session record, not the decomposition, does the work.

Why not just use a counting rule for fading, as classic tutoring systems do? This is a real threat. If dependence in practice tracks success and failure counts, a rule will match the controller at a fraction of the cost. The controller's claimed advantage is distinguishing repetitive from productive struggle, and frustration from confusion, using dialogue content. We expect the advantage to appear mainly for high-initial-dependence learners and on open-ended SciQ explanations, where success is not a binary signal. If the rule matches the controller there as well, we will recommend the rule and keep the simulator findings as the contribution.

Why not just make the voting protocol adaptive, or add an inertia agent to the committee? Adding a fifth voter leaves help level as an emergent outcome of each vote. The voting-protocol study already shows such outcomes shift with the protocol and flip in 56% to 70% of cases after peer review, which suggests they are unstable. We include a fifth-voter variant with the same cross-session record. If it matches the gated design, the gating architecture is unnecessary.

Why not just fine-tune one model on fine-grained intents? Intent Matters shows fine-grained intents help: 56.7% human preference over coarse ones. Its single model selects intents per turn without cross-session state. We include a fine-tuned 7B intent model as an additional arm on MathDial dialogues. The question is not whether it tutors a single problem well, but whether it lowers help across 40 problems.

Proposed Experiments

Simulator diagnostic

Run first, because it determines whether simulator results mean anything. For prompted personas, measure how much next-turn behavior changes when we alter the tutor's earlier help history versus when we alter only the persona's own previous utterance. If prompted behavior follows its own last turn rather than the tutor's history, existing simulator-based rankings are measuring LLM repetition rather than dependence.

Main simulated comparison

Run all baselines and ablations across three initial-dependence levels and several settings of the update rules. Report:

  • final unassisted accuracy;
  • total help consumed;
  • quit rate;
  • the help-level trajectory over the sequence;
  • Success@k against Telling@k;
  • tokens and latency per turn.

The decisive pattern for our claim has two parts. Under stateful students, the separated controller beats both ranked voting and the in-prompt ablation. The ranking of tutors also differs from the ranking obtained with prompted personas. A tutor ordering that is identical under both simulators would weaken the evaluation half of the claim.

Human check

A semester module in an introductory math or programming course, running about ten weeks so it spans the novelty-decay window, with three arms: separated controller, ranked voting, and rule-based fading. Participants are students who self-report lecture-dominated prior schooling. The primary outcome is an unassisted post-test. Help requests per problem over weeks and dropout are secondary outcomes. A study of this size can detect only moderate effects, so we treat it as a check of direction.

The human study answers two questions together. Does the controller's advantage survive with real students? Which simulator, stateful or prompted, predicted the human ordering of the three arms?