When a robot-run experiment goes wrong, the failure has one of three causes. The control script may have a bug, such as a wrong plate position or a reused pipette tip. The physical step may have failed, such as a viscous liquid that was not fully drawn up. Or the scientific guess behind the experiment may have been wrong. Today's autonomous labs mostly treat every bad cycle the same way. Either the optimizer that tunes numeric settings records a bad data point, or a language model is asked to "fix the code" and tries again. Both responses waste physical runs, and the waste gets much worse when the model is a small open-weight model on a single local GPU. Labs handling patient samples or proprietary chemistry often have to use such models, and these models tend to repeat the same fixes in circles. This proposal argues that the missing piece is a cheap step that runs before any repair: decide which of the three layers failed, then spend the local model's limited compute only on edits at that layer.
In two sentences, the idea is this. A lab-automation loop runs on local hardware. It first sorts each failure into code bug, physical-execution fault, or hypothesis miss, using symbolic checks and a small trained classifier. Only then does it let a local 7B model edit the script, and only at that layer, with rollback and a hard stopping rule. We expect this routing to recover most of the gap between a local model and a cloud frontier model on the metric that matters in a real lab: how many physical runs are wasted before the target is reached.
Core Claim
Our claim is that failure misattribution, not raw code-writing ability, is the main source of wasted cycles when small local models close the loop on laboratory scripts. Routing failures to the correct layer before repair should cut wasted physical executions at a fixed GPU-hour budget, and should do so by more than stronger fine-tuning or more refinement rounds achieve.
The evidence that motivates this comes from several directions. Refinement on execution feedback plateaus after a few rounds (Yan et al.). In Lean 4 proofs, VERINA found that 64 rounds of compiler feedback only raised proof success to about 20% while token use ballooned, because models kept looping on errors they could not diagnose. BioAgent Bench found two problems with local open-weight agents in bioinformatics:
- they trail cloud models (GLM-4.7 at 82.5% task completion versus 100% for Claude Opus 4.5);
- they fall into failure loops when the context fills with noise (a 28% drop in completion steps).
All three observations point the same way: undirected retries are expensive and plateau early, and on local hardware that cost cannot be absorbed.
Relation to Prior Work
The closest prior work is CellScientist. It poses a "refinement-routing problem" for virtual cell models: when a prediction disagrees with data, it sends the discrepancy to a specific node in a graph of biological assumptions and makes a local code edit there. That system is the source of our routing idea, but it differs from our setting in three ways. It works only on static datasets, it has no physical execution layer, and it assumes large compute. Our setting adds the layer CellScientist lacks, physical faults that look like scientific failures, and adds a budget constraint that makes routing mandatory rather than merely helpful.
Three systems handle lab scripts without any loop from execution back to code:
- ProtoAct turns written protocols into robot action primitives. It refines them before execution, using human notes and cloud models.
- PyLabRobot provides a hardware-agnostic Python interface for liquid handlers and a browser simulator, and shows GPT-3.5 can write scripts from natural language. It includes no debugging loop.
- MATTERIX builds a chemistry-aware digital twin, but its workflows are fixed state machines, and it explicitly leaves LLM-driven refinement to future work.
We reuse PyLabRobot as the execution layer, and we reuse the MATTERIX line of work as evidence that simulated physical feedback transfers to real hardware (90% sim-to-real success on pouring).
For the local repair model we draw on two methods. CoCoS shows that 1B models can learn to self-correct code through reinforcement learning with progressive pass-ratio rewards. This overcomes a "no-edit bias" that supervised fine-tuning leaves in place, where the model repeats its broken code verbatim. We adopt its training recipe, but move it from textbook Python to lab scripts and condition each repair on the routed failure layer. NESTER shows that a 7B local model does much better when a symbolic structure breaks the problem into pieces for it (70.7% top-1 on ManyTypes4Py). That result motivates our choice to have symbolic checks do the triage rather than asking the small model to diagnose raw error logs.
Two further pieces are borrowed as supporting machinery:
- rollback to the last stable script version after a regression (Fang et al.);
- evidence that 4-bit low-rank fine-tuning of 7B models on one 16GB GPU is competitive for domain tasks (Laboratory-Scale AI).
Research Program
The system has three parts.
The router. When a cycle ends badly, the router takes three inputs: the script, the execution trace, and the analytical readout (for example, absorbance from a plate reader). It assigns the failure to one of three layers.
- Code layer. A deterministic replay in the PyLabRobot simulator checks the deck-tree geometry, labware references, volume bounds, and tip state. If the replay fails, the fault is a code fault, and no neural judgment is needed.
- Execution layer. If the replay passes, a small classifier decides between a physical-execution fault and a hypothesis miss. It looks at execution-level signals: aspirated volume against commanded volume, pressure or liquid-level traces where the hardware provides them, and replicate disagreement within a plate. The classifier can be a second LoRA head on the same 7B base or a gradient-boosted model.
- Hypothesis layer. Only failures that pass both checks count as evidence about the science.
The repairers. Each layer has its own repair move.
- Code faults go to the local model, which proposes a patch. The patch is re-validated in the simulator before any hardware is touched.
- Execution faults go to the same model with a restricted edit space: liquid class, flow rate, mixing, tip depth, and retry of the step. The scientific design stays fixed, and the run is not recorded as a data point.
- Hypothesis misses go to a standard Bayesian optimizer, a statistical search method that proposes the next experiment from a model of past results. This data point is the only kind the optimizer sees.
The budget controller. A fixed ceiling on GPU-seconds and repair rounds applies per cycle. Every patch is committed as a version, with rollback when the pass ratio falls. Each round sees a context of only the current script, a routed summary of the diagnosis, and the last accepted patch. The model never sees the growing log that BioAgent Bench identifies as a failure trigger.
Training. The 7B model is fine-tuned with 4-bit low-rank adapters, first on synthetic script-repair pairs, then with the CoCoS reward on a fault-injected environment. The reward is computed on simulator pass ratio plus recovery of commanded volume.
The testbed. We will build a fault-injected testbed from the 22 cell-culture protocols and 910 executable subtasks of BioP2E, compiled to PyLabRobot. Faults are injected at all three layers:
- code faults are mutations of deck references, tip handling, and volume units;
- execution faults are aspiration shortfalls that depend on viscosity, set at rates in the range AutoBio reports for pipetting (42.7% success for the best vision-language-action policy);
- hypothesis faults come from synthetic assay response surfaces with a hidden optimum.
A real-hardware pilot on an Opentrons OT-2 then runs a dye-mixing color-matching task, with deliberately viscous glycerol stocks, to test transfer.
Baselines
Do-nothing reference. The script is written once and executed as is; failures are either logged as data or halt the run. This measures how many runs are lost with no repair at all. The ODACell platform's roughly 5% operational failure rate across 131 cells, in automated coin-cell assembly, shows that even well-engineered fixed pipelines leak runs this way.
Fixed scripts with Bayesian optimization. This is the classical closed loop: a hand-written script whose numeric settings a Bayesian optimizer tunes, with every outcome treated as scientific data. It is strong because it needs no language model and dominates in simple search spaces. MADE found that surrogate-based search wins in ternary spaces, and that LLM planners only pull ahead in quinary ones. It gets the same simulator and run budget as our system.
Unrouted local refinement. The same 7B model, trained with the same CoCoS recipe, receives the raw execution log and edits freely for up to the same number of rounds and GPU-seconds. This is the key ablation: it isolates routing from model quality. On textbook code, CoCoS reports a 35.8% improvement on MBPP with a 1.8% code-break rate. There is no reported result on Lacuna for any refinement method on lab-automation scripts, so this baseline's numbers will come from our testbed.
Strongest existing method: cloud frontier model with unrouted refinement. This models the approach of PyLabRobot's GPT-3.5 demonstration and ProtoAct's cloud pipeline, extended with a feedback loop. It is run only on non-sensitive synthetic protocols, as an upper reference. The closest reported agentic number is Claude Opus 4.5's 100% completion on BioAgent Bench, a different task. There is no reported result on Lacuna for cloud models closing the loop on robot scripts.
Ablations. We also test a router-only variant, in which symbolic checks run but repair is undirected, and a variant that swaps the learned execution-fault classifier for fixed thresholds.
What counts as winning. "Beating" a baseline means fewer wasted physical executions to reach the target at equal GPU-hours and equal run budget, with non-overlapping confidence intervals across at least 10 seeds per protocol. Wasted executions are runs made with a bug the system could have caught, or runs whose result was recorded under the wrong attribution.
Why not just…?
Why not just use a frontier cloud model? In many labs this is the honest answer. If data can leave the building, a frontier model may well do better. But the settings where autonomous labs handle patient-derived samples or proprietary formulations are exactly those where on-premise requirements (Large Language Models: A Comprehensive Guide for Radiologists) rule it out. There is also a cost effect: Laboratory-Scale AI measured roughly 10x cheaper inference for a local fine-tuned model, and that saving compounds over thousands of cycles. The experiment reports the gap directly, so readers can judge whether routing closes enough of it.
Why not just give the local model more refinement rounds? Prior work predicts plateaus and circular edits: Yan et al. on refinement plateaus, and VERINA's about 20% after 64 rounds of feedback. Our budget sweep will test this directly. If unrouted refinement matches routed refinement once given twice the rounds, the contribution shrinks to a compute saving, and we will report it as such.
Why not just run the PyLabRobot simulator as a validator and skip the language-model repair? This is a real threat, and it forced a change to the design. A deterministic replay will probably catch most code-layer faults on its own, so in our design it does exactly that job, and the language model receives no credit for code-level catches the checker makes. The open question is whether execution faults and hypothesis confusion also need learned handling. The per-layer breakdown settles it. If the checker plus Bayesian optimization matches the full system on execution-fault protocols, the learned components are unnecessary.
Why not just use a fixed script with a Bayesian optimizer? It cannot change procedure: it cannot switch liquid class, add a mixing step, or reroute labware. Worse, it quietly records physical faults as scientific data, which corrupts its own model of the search space. We measure this corruption directly as the distance between the optimizer's inferred optimum and the true optimum of the synthetic response surface.
Proposed Experiments
Routing is detectable. The first check is router accuracy on held-out injected faults, broken down by layer. If execution faults and hypothesis misses cannot be told apart at better than chance from the available signals, the idea fails at its first step.
Routing saves runs. The central experiment compares wasted executions, cycles to reach the target, and GPU-seconds per successful cycle across all baselines on the BioP2E-derived testbed. It sweeps fault mix and repair budget.
Routing keeps the science clean. We compare the optimizer's final estimate of the optimum with and without routing. This measures how much misattributed failures bias scientific conclusions, which matters independently of efficiency.
The result transfers to real hardware. The OT-2 color-matching pilot reports the same metrics over a small number of campaigns. Its purpose is to confirm the sign of the simulated effect, not to establish its size.
Robustness. Because BioAgent Bench measured a mean run-to-run overlap of only 0.43, all local-model results are reported with variance across seeds.
References
- CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
- VERINA: Benchmarking Verifiable Code Generation
- BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics
- CellScientist: Dual-Space Hierarchical Orchestration for Closed-Loop Refinement of Virtual Cell Models
- ProtoAct: Turning Wet-Lab Protocols into Embodied Robotic Actions
- PyLabRobot: An open-source, hardware-agnostic interface for liquid-handling robots and accessories
- MATTERIX: toward a digital twin for robotics-assisted chemistry laboratory automation
- Self-Correcting Code Generation Using Small Language Models
- Dataflow-Guided Neuro-Symbolic Language Models for Type Inference
- MEIC: Re-thinking RTL Debug Automation using LLMs
- Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings
- AutoBio: A Simulation and Benchmark for Robotic Automation in Digital Biology Laboratory
- Automated electrolyte formulation and coin cell assembly for high-throughput lithium-ion battery research
- MADE: Benchmark Environments for Closed-Loop Materials Discovery
- Large Language Models: A Comprehensive Guide for Radiologists