AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses

Cheng Qian, Wenting Zhao, Liangwei Yang, Heng Wang, Jielin Qiu, Heng Ji, Silvio Savarese, Huan Wang, Shelby HeineckearXiv 2026 · arXiv

TL;DR

Instead of updating model weights to transfer capabilities from large to small language models, why not let a powerful model build a custom inference-time environment for a weaker one? This paper introduces "strong-to-weak scaffolding," where a strong "builder" model automatically designs a persistent harness (combining prompts, routing logic, and deterministic code) to optimize a fixed "target" model. Without any parameter updates, this test-time capability transfer nearly doubles the target model's performance on complex reasoning tasks.

What Problem Does This Paper Address?

Traditional model distillation transfers capabilities from a large "teacher" to a smaller "student" by modifying the student's weights during training. While effective, this requires expensive retraining and parameter updates. On the flip side, inference-time techniques like Chain-of-Thought or self-consistency optimize how a single model handles a specific prompt, but they don't create reusable, cross-model structures.

The authors explore a compelling middle ground: strong-to-weak scaffolding. When a small model fails a task, it might not be due to a complete lack of capability, but rather excessive cognitive load. Can a stronger model act as a "system architect," engineering an external harness (routing logic, verification checks, deterministic code solvers) that reduces this cognitive load and allows the weaker model to succeed?

Methods

The authors formalize a two-stage automatic harness-building framework:

  1. Stage 1: Recursive Scaffold Building: A strong builder model ($M_{\text{build}}$) is given a task description, a demonstration, and a small validation set (just 5% of the total data). The builder iteratively writes, tests, and refines a Python-based scaffold to maximize the target model's ($M_{\text{tar}}$) accuracy on the validation split.
  2. Stage 2: Hidden Evaluation Testing: The finalized scaffold is exported as a black-box entry point and evaluated on a completely hidden test set ($T$) using the target model. The target model's weights remain completely frozen throughout.
Figure 1: Overview of the Strong-to-Weak Scaffolding evaluation framework. During recursive scaffold building, the builder model iteratively refines a scaffold to improve target-model performance on validation sets. During hidden evaluation testing, the final scaffold is evaluated on the full hidden test set to measure target-model performance.

The builder is free to implement any combination of techniques, including benchmark-aware routing, prompt templates, strict answer-format enforcement, and even offloading sub-problems entirely to deterministic symbolic code.

Datasets

The framework is stress-tested on the demanding domain of Theory-of-Mind (ToM) reasoning, aggregating four prominent benchmarks into a 3,900-item hidden test set:

  • BigToM: Binary questions tracking whether an agent observed a world change.
  • Hi-ToM: Higher-order recursive belief challenges (up to recursion depth 4) involving deception and object tracking.
  • MMToM-QA: Binary Bayesian goal and belief inference from action traces.
  • MuMA-ToM: Multi-agent, multi-choice questions evaluating social goals and nested beliefs from text dialogues.

Key Insights and Ideas

  • The Headroom Law: Scaffolding acts primarily as a competence-recovery mechanism. The realized performance uplift is strongly predicted by how much "headroom" (1 - baseline accuracy) the target model leaves on the table. It helps most where the target model has latent abilities but fails to deploy them reliably.
  • Scaffolding Can Backfire on Strong Targets: If a target model is already performing near its ceiling, adding complex external routing or rules can disrupt its natural capabilities, causing a regression in performance.
  • Disciplined Engineering Beats Exotic Prompting: The most successful scaffolds rely on universal reliability controls—like strict format enforcement, greedy decoding (temperature 0), and task routing—combined with deterministic offloading (writing code to handle predictable sub-problems), rather than simply forcing the target model to write longer explanations.
  • Compute for Thinking, Not Probing: Interestingly, the number of times a builder queries the validation set does not correlate with final test success. What does matter is the builder's internal reasoning effort; stronger builders create better scaffolds through deeper initial hypothesis generation rather than brute-force trial and error.

Key Results

The experimental setup evaluated various builder models (including GPT-5.5, Claude 3 Opus/Sonnet, Gemini, and Grok) scaffolding a weak target model, primarily GPT-5.4-mini.

Main results across different builder models and platforms.
  • Massive Performance Gains: The best automatically constructed scaffold (built by GPT-5.5 on the GPT Codex platform) raised GPT-5.4-mini's macro-average accuracy from a baseline of 0.49 to 0.91—an absolute gain of 42%.
  • Beating Larger Models: The scaffolded mini model frequently outperformed much larger, un-scaffolded models (such as vanilla GPT-5.4 or GPT-OSS-120B) without changing a single weight.
  • Monotonic Improvement with Effort: When scaling the builder's internal reasoning effort (from low to extra-high), the quality of the resulting scaffold improved monotonically, showing that more builder deliberation directly translates into cleaner, more robust harness code.

Limitations and Future Work

While automated scaffolding achieved near-ceiling performance on highly structured tasks like BigToM, clear performance gaps remain on more fluid, ambiguous tasks. The scaffolds struggled to compile away errors in deep belief recursion (Hi-ToM depth 2+) and complex Bayesian goal inference from action traces. In these regimes, the harness is forced to hand the reasoning back to the weak model, where it remains error-prone.

The authors suggest that future work should expand this paradigm to broader benchmark families containing different mixtures of symbolic structure and open-ended reasoning. Additionally, this setup can serve as a novel benchmark to evaluate frontier models not just on how well they solve a task, but on how effectively they can teach and structure an environment for a weaker model to solve it.

Summary updated

Abstract

Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter's parameters, through teacher forcing, on-policy distillation, and related training-time methods. In this paper, we ask whether such transfer can instead occur at test time. We study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably without any parameter updates. Using four representative Theory-of-Mind benchmarks, each builder model uses 5% of the data as a validation set to iteratively refine its harness over multiple rounds, after which the finalized harness is evaluated on the full test set. Empirically, this form of test-time capability transfer is highly effective, nearly doubling average target-model performance from 0.49 to 0.91. Our analysis shows that the gains come primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement, rather than from encouraging the target model to reason more extensively or sample more broadly. We further find that builder-model reasoning effort improves harness quality monotonically, platform effects are modest relative to the builder model's own capability, and weaker target models receive the largest gains. These results suggest that inference-time harness design is an important complement to conventional training-time distillation, enabling strong models to transfer cognitive structure to weaker models without retraining.
BibTeX
@misc{qian2026ai,
  author = {Cheng Qian and Wenting Zhao and Liangwei Yang and Heng Wang and Jielin Qiu and Heng Ji and Silvio Savarese and Huan Wang and Shelby Heinecke},
  title = {{AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses}},
  year = {2026},
  eprint = {2608.12307},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.12307},
}
Figure