Low-Rank Judges That Disagree With Their Thinker

Multi-agent reasoning systems split a hard problem among several roles. One agent proposes intermediate steps, another checks them, and a third computes. The checking role is the one that matters most: a model asked to grade its own reasoning tends to approve it, and a separate critic is supposed to break that confirmation bias. On small open models this design falls apart. Prompted personas blur together, and the "critic" approves what the "thinker" wrote because both are the same weights reading the same context. We propose to make the critic a separately trained low-rank adapter on a shared frozen backbone. A low-rank adapter, or LoRA, is a small add-on weight matrix, under 1% of the model's parameters, that changes the model's behavior. The critic adapter would be trained specifically on the thinker adapter's own current mistakes. We would measure directly whether this produces a judge whose errors are independent of the thinker's, and at what serving cost.

In two sentences: a verifier only helps if it fails on different inputs than the generator, and on a single small backbone that independence has to be trained into the weights, because prompting does not supply it. We build a thinker–judge–executor loop for math reasoning in which each role is a low-rank expert. The judge is retrained against the thinker's evolving failures. The role adapters are placed so that switching roles reuses the shared attention cache, which makes the loop nearly as cheap as one model talking to itself.

Core Claim

The claim is that, for a fixed small backbone and a fixed inference-token budget, a judge expert trained on the thinker expert's on-policy errors will reject a larger fraction of the thinker's wrong intermediate steps than a prompted self-judge or a jointly trained single adapter. It should do so while keeping its false-rejection rate comparable, and the gain should translate into end-task accuracy that beats sampling-and-voting at matched compute. The quantity we care about is the judge's miss rate on wrong steps, $m = P(\text{judge accepts} \mid \text{step is wrong})$. We compare $m$ against a reference judge taken from a different model family, which approximates what independent verification looks like. The second claim is architectural. If role adapters are confined to deeper-layer feed-forward and output projections and leave the key and value projections untouched, every role can reuse the same cached context. Role switching then costs no re-reading of the shared transcript, and we test whether this placement costs accuracy.

Where This Sits

The multi-agent side supplies the motivation and the workflow. MACM splits math solving into a Thinker that proposes new conditions, a Judge that admits only verified conditions to a shared list, and an Executor that computes. With GPT-4 Turbo it lifts Level-5 MATH accuracy from 54.68% to 76.73% and reaches 99% on the 24-point game. MACM runs entirely by prompting a frontier model. We keep its loop and ask whether the loop survives on 4B–8B open models when roles live in weights rather than prompts. Self-collaboration Code Generation via ChatGPT runs an Analyst–Coder–Tester team and also reports the constraint that motivates us: below roughly 13B parameters, models fail to hold distinct personas. Critic-style loops in ReChisel and LEGO point the same way. Separate verification helps when the verifier is genuinely separate.

Two papers already put roles into low-rank experts, and they set the bar for novelty. MoRAgent tunes one model with Reasoner, Executor and Summarizer LoRA clusters selected by a role-aware gate, with an orthogonality loss between adapters. Its pipeline is forward-only and has no verifying role, so nothing in it tests whether parameter separation yields independent criticism. MoRSE decomposes coding tasks into a graph of subtasks served by one role expert plus routed subtask experts, trained with a two-level reinforcement-learning credit scheme. It uses only "execute" and "merge" roles, names Reviewer and Refiner roles as future work, and evaluates only on code. Adding a reviewer expert to MoRSE would be the obvious next step. Our proposal differs in three specific ways. The judge is trained adversarially against the thinker's current error distribution rather than as one more role under a shared objective. The primary outcome is a measured error-independence statistic rather than only task accuracy. The adapter placement is chosen to make role switching cache-compatible. We work on math, where MoRSE has no results.

The adapter-mixture literature constrains the design. LoRALib shows that automatic routing among LoRA experts lags manual selection on math and logic. We therefore do not learn a router: the workflow itself says which role is speaking, which removes the selection-gap failure mode by construction. LoRA-Mixer finds that experts placed on the attention projections, including keys and values, work best, which is in direct tension with our cache-sharing placement. We treat that as an empirical question, below. LoRASA supplies the method hint that agent-specific low-rank adapters are most useful in deeper layers, though only in reinforcement-learning control. MoRAgent reports that roles need asymmetric capacity, which we test for the judge. Mixture-of-Agents shows that conditioning on other models' outputs helps, but it does so with several full models and no shared weights. It is the expensive upper reference for "independent agents."

Research Program

The system is one frozen backbone, Qwen3-4B and Llama-3.1-8B (the backbones MoRSE used), carrying three LoRA experts: Thinker, Judge and Executor. They run in MACM's loop. The Thinker proposes a condition, the Judge accepts or rejects it with a short reason, accepted conditions join the list, and the Executor computes when the list suffices. A fixed controller decides which expert is active, so there is no learned gate.

Training proceeds in rounds. In round zero, Thinker and Executor are fine-tuned on condition-mining trajectories for MATH and 24-point training problems. Correctness labels for every intermediate condition come from exact checking on the 24-point game and from a larger teacher model on MATH. The teacher labels are a known source of noise, so we audit a sample by hand. The Judge is then trained on the current Thinker's own proposed conditions, labeled correct or incorrect. In each later round the Thinker is updated on loop outcomes, its new failures are collected, and the Judge is retrained on them. This keeps the Judge aimed at where the Thinker currently goes wrong rather than at a stale error distribution. The analysis centers on the miss rate $m$ and the false-rejection rate, broken down by error type: arithmetic slips, misread conditions, and invalid inference. The guiding hypothesis is that a low-rank judge can decorrelate from execution-style errors but not from errors rooted in what the shared backbone does not know. If that holds, it defines the limit of what same-backbone multi-agent systems can achieve.

Baselines

The naive reference is the backbone answering with plain chain-of-thought, together with the same backbone running MACM's loop purely by prompting, with no adapters. This reproduces the regime in which Self-collaboration reported persona collapse below 13B. No reported result was found on Lacuna for MACM on 4B–8B open models, so we establish this number ourselves. Beating it shows only that training helps, not that separate roles help.

The strongest compute-matched competitor is self-consistency: the backbone fine-tuned with one LoRA on correct solution traces, sampled repeatedly and majority-voted. The number of samples is chosen so that total generated tokens match the loop's average. We consider this the real threat, because voting also exploits error independence, only across samples rather than across roles. We count a win only if the loop beats voting at equal tokens on MATH Level 4–5.

The strongest existing parameter-efficient multi-role method is a MoRAgent-style mixture-of-roles: role LoRA clusters with orthogonality and balance losses, extended with a judge role trained once on round-zero data. It is the closest published design, and reported gains for it include a 12.0% MATH improvement on Qwen2.5-1.5B-Coder. That paper does not report Level-5 MATH or judge statistics. We run it on the same data and the same total trainable parameters, with the same hyperparameter search budget of twelve configurations per method. As the frontier-prompted ceiling we cite MACM's GPT-4 Turbo Level-5 result of 76.73%, for context rather than as a matched comparison.

Three ablations isolate the key ingredients. First, a single shared LoRA trained on all three roles, distinguished only by a role token. This tests whether separate parameters matter at all. Second, a judge trained once instead of in rounds. This tests whether the on-policy targeting matters. Third, judge adapters on query, key and value projections versus confined to deep-layer feed-forward and output projections. This tests whether cache sharing costs accuracy, which is LoRA-Mixer's finding turned into a falsifiable check. A reference judge from a different model family of similar size estimates the miss rate that true independence would achieve.

Why not just…?

Why not just prompt the same model to act as a critic? Prompted self-judging reads the same context with the same weights, and Self-collaboration already reports that sub-13B models do not sustain distinct roles. MoRSE's prompt-only system reached 72.5% execution on SRDD against 86.25% with trained experts. That is code, not math, so our prompted-MACM baseline will show whether the gap carries over. The direct evidence will be the miss rate $m$: if the prompted judge's $m$ is near the trained judge's, our claim fails.

Why not just train one LoRA on all roles? It is cheaper, and with a role token it might separate behaviors well enough. Single-adapter training is reported to underperform mixtures: MiLoRA reports 61.5% versus 58.4% for standard LoRA on reasoning, and LoRA-Mixer reports +3.79% over LoRA on GSM8K. Neither number concerns verification, though. The shared-adapter ablation answers the question directly. If it matches the separate judge on $m$, separation is unnecessary, and we would report that.

Why not just sample more and vote? This is the most serious alternative. Voting needs no judge and also benefits from decorrelated errors. Voting cannot reject a wrong step that most samples share, while a judge targeted at the thinker's systematic errors can. Our prediction is therefore that the loop's advantage concentrates on problems where the voting majority is wrong. We test this by stratifying MATH problems on whether majority voting fails. If the loop does not beat voting at matched tokens, the idea does not survive in its current form.

Why not just use a separate full model as judge, or Mixture-of-Agents? That probably yields more independent errors, and we include it as the reference judge. It doubles memory and loses the shared cache, though. The question is how much of its independence a low-rank judge recovers at a fraction of the cost. We report accuracy against peak memory and tokens per second for both.

Proposed Experiments

The mechanism study uses the 24-point game, where every proposed condition can be checked exactly. We plot the Judge's miss and false-rejection rates across training rounds, against the prompted judge, the shared-adapter judge and the cross-family judge. The main evaluation is MATH by difficulty level, with GSM8K as a sanity check that easy-problem accuracy is not harmed. As a transfer probe we run SciCode subproblems with the condition list replaced by verified helper functions. On SciCode, frontier models reach about 28% on isolated subproblems but 7.7% on main problems (o1-preview), which is the integration gap a judge should narrow. We report accuracy, the full token count per problem, wall-clock latency with and without cache sharing, and trainable parameter count.

Three checks are decisive. The trained Judge's miss rate must fall well below the prompted judge's and approach the cross-family judge's. The loop must beat matched-token voting specifically on problems where voting's majority is wrong. Confining adapters to cache-compatible layers must lose under one point of MATH accuracy while cutting role-switch cost. If the first check fails, parameter separation does not buy independent verification on a shared backbone. That negative result would also bear on MoRSE and MoRAgent. The breakdown by error type would then show which failures a same-backbone system can never catch.