Evidence-Gated Influence: Letting Weak Agents Help Strong Ones Without Talking Them Down

When a strong language model debates a weaker one, the pair often does worse than the strong model alone. The weaker model's confident but wrong arguments pull the stronger model off its correct answer. Prior work calls this the distraction effect. Yet weaker models from other model families also catch errors the strong model misses, and that diversity is exactly what makes heterogeneous ensembles worth running. This proposal tests one change. The weak agent may still change the strong agent's mind, but only by producing a concrete, checkable artifact: a disputed span of text, a recomputed number, a re-extracted chart value. That artifact is then confirmed or refuted by an independent check that neither agent controls. The strong model never reads the weak model's arguments, confidence, or prose. Influence passes through verified evidence and never through persuasion.

The idea comes from scientific practice, where a surprising result is trusted only if a second, independently built pipeline reproduces it. We turn that practice into an interaction protocol between unequal agents. The core prediction can be stated in two sentences. Under free-form debate, the rate at which a strong model abandons a correct answer rises with the capability gap. Under evidence gating, that harmful rate should stay roughly flat as the gap grows, while most of the beneficial corrections the weak model enables are kept.

Core Claim

Capability-mismatched interference is a property of the communication channel, not of heterogeneity itself. Two kinds of stance change can be separated using ground truth. A harmful flip is a change from a correct to an incorrect answer. A helpful flip is a change from incorrect to correct. Writing $H(g)$ for the harmful flip rate and $B(g)$ for the helpful flip rate of the strong agent at capability gap $g$, measured as the difference in standalone accuracy, we claim the following. In free-form debate $H(g)$ grows with $g$, which is what Examining Inter-Consistency of Large Language Models Collaboration and Towards Scalable Oversight with Collaborative Multi-Agent Debate in Error Detection report qualitatively. Under evidence gating, $H(g)$ is approximately constant in $g$ and $B(g)$ is largely preserved. The net quantity that matters is

$$\Delta(g) = B(g)\,(1 - a_s) - H(g)\,a_s,$$

where $a_s$ is the strong agent's standalone accuracy. $\Delta(g)$ is the accuracy change the partnership adds over the strong model alone. The opening question of this direction is when $\Delta(g) > 0$ for large gaps.

Relation to Prior Work

Three papers motivate the gap by documenting the failure.

Examining Inter-Consistency of Large Language Models Collaboration runs debates between models of different tiers on seven commonsense benchmarks. It identifies both a dominance effect and a distraction effect, and finds that a GPT-4 judge over the transcript adds only about 0.6% accuracy.

MAMM-Refine shows that mixing model families improves faithfulness. It also reports that a weaker critic corrupts refinement, and it concludes that systems need capability parity.

The Collaboration Gap shows that strong models imitate weaker partners' sloppy communication conventions. It also shows that who speaks first decides whether the pair recovers.

All three paper treat the shared conversation as fixed and measure what happens inside it. We change the channel instead.

One paper is the closest competitor. ColMAD reframes debate as mutual auditing, with exact-quote evidence and calibrated confidence, and gains 4% over the best single agent on ReaLMistake. Its authors still report degenerated collaboration when Mistral-7B is paired with frontier models. Our protocol differs from ColMAD in a specific way. ColMAD's evidence requirement is a prompt instruction inside a shared dialogue, so the strong model still reads the weak model's framing and confidence. In our protocol the strong model sees only an artifact, and only after a mechanical check has decided whether that artifact holds. Isolating that difference is the point of our main ablation.

Two papers supply the method.

BeaconChart uses a verifier that re-extracts chart fields without seeing the actor's workspace. This raises the probability of identifying an error, given that the answer failed, from 27.95% to 54.15%. However, it uses a single model family, Qwen3-VL-32B, and never asks what happens when the verifier is much weaker than the actor.

AutoCode and MathHay certify outputs by requiring two independently written programs to agree exactly. We reuse that dual-execution idea as the gate. Neither paper places asymmetric agents in interaction.

One paper constrains the design. A Diagnostic Study of Multi-Agent LLMs for Real-World Debates finds that high-quality deliberation recovers the correct answer 74% of the time when the initial majority is wrong. This is a real point in favor of open dialogue, and we must not throw that benefit away. Our helpful-flip measurement $B(g)$ is designed to detect exactly that loss.

An Empirical Study of Group Conformity in Multi-Agent Systems shows that more capable agents induce more conformity (74.33% versus 39.83%). This predicts that persuasion, and not content, drives stance changes, and that prediction is testable here.

Research Program

The protocol has three stages.

  1. Blind solving. The strong and weak agents each solve the task without seeing each other.
  2. Witness production. If their answers differ, the weak agent is asked for a witness, meaning a typed, checkable artifact pointing to where the strong answer goes wrong. The witness type depends on the domain. On ReaLMistake error detection it is a quoted span plus the specific claim it violates. On MathHay it is a sub-quantity with the Python expression that computes it. On ChartQA and ChartX it is a named data field and its value.
  3. Gating. An independent checker evaluates the witness. Depending on the domain, the checker is a code interpreter, a fresh blind re-extraction by a third model call, or an exact string match against the source. Only confirmed witnesses reach the strong agent, stripped of the weak agent's wording. The strong agent may then revise its answer.

We will instrument every run to classify each stance change as harmful or helpful. We will also compute the Influence Asymmetry and Responsiveness metrics from the diagnostic study, so that we can ask whether gating changes deliberation health, not just accuracy.

Capability gap will be varied systematically by pairing one strong model with weaker partners of decreasing size. The text pool will include frontier models with Llama-3.1-70B and Mistral-7B, the pairings ColMAD found problematic. The chart pool will use Qwen3-VL-32B with a much smaller sibling. Commonsense benchmarks from the FORD study serve as a negative control: they admit no mechanical check, so gating should give no benefit there. That marks the method's boundary honestly instead of hiding it.

Baselines

The naive reference point is the strong model alone, given the same number of model calls as the full protocol through self-consistency sampling. This is the bar every partnership must clear, because the failure we study is falling below it. Beating it means $\Delta(g) > 0$ at the largest gap.

The strongest existing method is ColMAD run on ReaLMistake with the same model pairs. We will follow its published prompts and use the same number of strong-model calls and the same tuning budget as our protocol. The only reported result found on Lacuna is relative: ColMAD improves 4% over the best single-agent method and 19% over competitive debate. No absolute ReaLMistake score for ColMAD was found in the file, so we will reproduce it ourselves. Beating ColMAD means higher error-detection accuracy and recall at large gaps, with $H(g)$ significantly lower and $B(g)$ not significantly lower.

On chart tasks, the strongest reference is BeaconChart, which reports 91.60% on ChartQA and 72.05% on ChartX. We will rerun its blind-verifier design with a weak verifier, matching models and calls, to see whether its uplift survives asymmetry.

On MathHay, the reported reference is Gemini-1.5-Pro-002 at 51.26% accuracy at 128K tokens. We will instead compare our strong model alone against our strong model under the same context lengths.

The ablation that isolates the key ingredient is three-way, with the same model pairs, the same witnesses, and the same calls in every arm. In the first arm the weak agent's witness is delivered as its full free-text argument. In the second arm the witness alone is delivered without any gate. In the third arm the witness is delivered only after the independent check passes. Comparing the first and second arms isolates rhetoric. Comparing the second and third isolates verification.

A fourth control delivers only a content-free disagreement flag, after which the strong model re-checks its own work. This control tells us whether the weak agent's content matters at all.

Why not just…?

Why not just use the strong model alone? For large gaps this may honestly be the right answer, and it is our naive baseline. The case against it is indirect but consistent. Evaluating LLMs at Detecting Errors in LLM Responses finds that frontier models have very low recall when detecting errors, and that self-consistency and majority voting do not help on ReaLMistake. This suggests the strong model's blind spots are correlated across its own samples. If $\Delta(g) \le 0$ under gating at every gap tested, we will report that heterogeneous partnerships are not worth running for these task types.

Why not just run both models blind and let the strong model win ties, BeaconChart-style? This is the most serious threat, because our protocol contains it. The disagreement-flag control is exactly this cascade: the weak model's disagreement triggers re-verification without supplying any content. If the flag control matches full gating, the contribution shrinks to "disagreement is a useful trigger." That would still be a finding about asymmetry BeaconChart did not test, but the witness mechanism would be unnecessary, and we would say so. Our expectation is that witnesses beat the flag because they tell the strong model where to look. On MathHay, accuracy collapses from about 70% on one-step to below 30% on three-step problems, so locating the faulty step should matter most there.

Why not just prompt the strong model not to concede? Evaluating LLMs at Detecting Errors in LLM Responses reports that prompt variations give no significant gains for strong models. The conformity study finds that flipping the framing does not remove conformity. More importantly, stubbornness lowers $H$ by also lowering $B$. We include a "hold your position unless clearly shown wrong" prompt arm and report both rates, so the trade-off is visible rather than assumed.

Why not just weight votes by capability? Weighted voting can never adopt a correct minority answer from the weaker model. That makes $B(g) = 0$ by construction whenever the strong model is outvoted only by itself. The conformity results also show that capability-weighted authority is exactly the dynamic that suppresses correct dissent.

Proposed Experiments

The decisive experiment plots $H(g)$ and $B(g)$ against capability gap for free-form debate, ColMAD, and gated influence on ReaLMistake, MathHay, and ChartQA/ChartX. The hypothesis is confirmed if three things hold. First, the slope of $H$ in $g$ is significantly positive for debate and ColMAD and near zero for gating, tested with a logistic regression of flip outcome on gap with protocol interaction. Second, gating keeps at least most of debate's $B(g)$. Third, $\Delta(g)$ for gating stays positive at the largest gap where debate turns negative.

The hypothesis is refuted if gating cuts $H$ only by suppressing all stance changes, or if the disagreement flag alone matches it.

A secondary check uses the FORD commonsense benchmarks, where no mechanical gate exists. We expect gating to fall back to roughly strong-model-alone performance there, which confirms the mechanism is the check and not the protocol's structure.

A third check computes Influence Asymmetry from the diagnostic study across all arms. The test is whether gated runs show influence that tracks which agent was correct rather than which agent was stronger. That would be direct evidence that persuasion has been replaced by evidence.