Code-Refining Agents That Cannot Fool Themselves on Noisy Cryo-EM Data

Cryogenic electron microscopy (cryo-EM) recovers the 3D shape of a protein by averaging hundreds of thousands of extremely noisy 2D images, often with less than one part signal to a hundred parts noise. In this regime the reconstruction code itself determines what you see: its pose-search schedule, regularizers, and loss terms. Large language model (LLM) agents that repeatedly edit, run, and score programs have recently learned to improve whole solvers, not just fill in their parameters. Pointing such an agent at cryo-EM reconstruction code is an obvious move. Done naively, it is also dangerous. Every cryo-EM quality score computable without ground truth is itself a noisy estimate. An agent that keeps a code edit whenever that score goes up will eventually select programs that are good at producing confident-looking structure from noise.

This note proposes an execution-guided code refinement loop for low-SNR reconstruction whose central design choice is the acceptance rule. The agent never sees the data used to decide whether an edit is kept. A sealed particle split and a noise-only control act as a rollback oracle. We test two claims. The first is that agent-edited reconstruction programs beat the strongest ab initio method and a matched hyperparameter search on the CryoBench noise ladder. The second, which we expect to be the more important finding, is that without the sealed rule the same loop measurably overfits its own score.

Core Claim

Iterative code refinement assumes the execution environment is a trustworthy oracle. A compiler, a test suite, or an optimality gap gives the same verdict every time. Low-SNR inverse problems break that assumption. The score is a random variable, and selection pressure exploits it. We claim three things. A refinement loop for cryo-EM code must treat its fitness function as a resource that gets used up. Accepting an edit should require improvement on particles the agent has never been scored on, plus a null result on pure-noise input. Under this rule, program-level edits can close part of the gap where current ab initio methods fail: severe noise combined with heterogeneity.

We quantify the danger with a gaming gap. Let $S_{\text{vis}}$ be the score the agent optimizes, which is Fourier Shell Correlation (agreement between two independently reconstructed half-maps, frequency by frequency) on data it is scored on. Let $S_{\text{true}}$ be the per-image FSC against the known ground-truth volume, which is available on synthetic benchmarks. Then

$$G = S_{\text{vis}} - S_{\text{true}},$$

and the prediction is that $G$ grows with the number of accepted edits under a visible-score rule and stays flat under the sealed rule.

Why This Gap, and What Prior Work Says

The imaging side supplies the problem. CryoBench built synthetic datasets with ground truth, including an IgG antibody whose Fab arm rotates (IgG-1D) at three noise levels. It reports that ab initio methods "struggled with very low SNR" on IgG-1D-noisiest, while CryoDRGN-AI "generally outperformed other ab initio methods." CryoDRGN-AI couples a coarse hierarchical pose search with gradient refinement of poses and a neural field. Its search heuristics and losses are fixed code, which is exactly the object we propose to edit.

CryoSplat represents the volume as 3D Gaussians and converges in 5 epochs with under 380 MiB of GPU memory. We reuse it for a different reason than its accuracy: it makes each candidate program cheap enough to run many times. CESPED shows that learned pose predictors still leave a 1.3–3.4 Å resolution gap. This motivates editing the search procedure rather than replacing it with a learned predictor. The internal "recycling" loops in ModelAngelo and CryoACE feed coordinate estimates back into their networks. They refine numbers, never code, so they sit beside this proposal rather than competing with it.

The code side supplies the method and the warnings. A2DEPT evolves complete solver programs in a tree with annealing-style selection, and even beats LSODA on stiff differential equations. We adopt its program tree and self-healing step, but its fitness is a deterministic optimality gap. MEIC separates a Debug Agent from an independent Scorer Agent and reverts any patch that regresses. We reuse this structure, but make the scorer blind to the agent's data. POLO optimizes runtime, which is measurable without noise. Its profiling-guided focus on bottleneck functions informs which modules we expose for editing.

Two papers constrain the design. Yan et al. find that feedback gains plateau after a few iterations, so we budget a small number of accepted edits. Srivastava et al. show that iterative loops amplify flaws while satisfying the test. This is the code-security analogue of the score gaming we expect, and it is why the noise-only control exists.

Research Program

The editable substrate is the CryoDRGN-AI codebase for the heterogeneous track. A CryoSplat-style homogeneous reconstructor serves as a fast development track. The agent may edit four modules: the pose-search schedule, the loss and regularization terms, the latent-variable prior, and the frequency-marching schedule. The image-formation physics and the evaluation code are read-only. This prevents the agent from rewriting the forward model or the scorer.

Particles in each dataset are split three ways before the loop starts.

  1. The agent split is the data the agent runs on. It sees half-map FSC, logs, and tracebacks there, in the CAAFE-style traceback loop of CAAFE.
  2. The sealed split is used only by the scorer. The scorer returns a single accept or reject bit and adds noise to its threshold, so repeated queries leak little information.
  3. The reporting split is never touched until the end.

The accept rule has two conditions. An edit is accepted only if half-map FSC improves on the sealed split. It must also fail to produce above-chance FSC when run on a matched pure-noise particle stack seeded with the current reference. Rejected edits roll back, MEIC-style. To address the partial-migration bugs reported by A2DEPT, any edit touching a shared data structure triggers static dependency checks and a short smoke run before it is scored.

Baselines

The do-nothing reference is CryoDRGN-AI at its released defaults, run on the identical splits. It is also the strongest existing method, since CryoBench reports it generally outperforms other ab initio methods. We also include RECOVAR, which CryoBench reports strong on IgG-1D, as a non-ab-initio reference. We found no reported per-dataset number on Lacuna for CryoBench IgG-1D-noisiest; the summary only states that ab initio methods struggled. We will therefore rerun both methods ourselves under the CryoBench per-image FSC protocol. Beating CryoDRGN-AI means a higher per-image FSC on the reporting split of IgG-1D-noisiest, with the difference exceeding the spread across three seeds.

For the real-data check on CESPED targets, the reference point is the paper's Image2Sphere result: 28.7° mean angular error and a 1.3–3.4 Å resolution gap to classical global search. On EMPIAR-10049, the homogeneous reference is CryoSplat's reported 2.49 Å versus about 4 Å for CryoSPARC and CryoDRGN.

The matched hyperparameter search gets the same GPU-hour budget and the same sealed acceptance rule. It runs Bayesian optimization over every numeric option the codebase exposes. This isolates whether rewriting code adds anything beyond tuning numbers.

The visible-score ablation is the identical agent loop with the sealed split and noise control removed. It accepts edits on the agent split's own FSC, like a direct A2DEPT transplant. It isolates the proposal's key ingredient.

Every arm receives the same compute budget, the same LLM, and the same number of candidate executions.

Why not just…?

Why not just tune hyperparameters instead of editing code? This is the real threat. If Bayesian optimization under the same sealed rule matches the agent, the conclusion is that cryo-EM failure at low SNR is a tuning problem, and we will report it as such. The matched-search baseline exists to settle this. We expect code edits to matter specifically where the defect is structural: for example, a pose search that commits to coarse poses before heterogeneity is resolved. No numeric knob can reorder that.

Why not just use gold-standard half-map FSC as the agent's score, as the field already does? Half-map splitting protects a single reconstruction against overfitting. It does not protect a search over hundreds of programs that are each selected on the same halves. The visible-score ablation measures the resulting gaming gap $G$ directly, using CryoBench ground truth. If $G$ stays flat without sealing, the extra machinery is unnecessary and we will drop it.

Why not just replace pose search with a learned pose predictor? CESPED shows that the best supervised predictor still leaves 28.7° mean error and a resolution gap of up to 3.4 Å. Learned poses are a warm start, not a substitute, so the agent is allowed to call such a predictor inside its edited search.

Proposed Experiments

The primary experiment runs all arms on the CryoBench noise ladder: IgG-1D, IgG-1D-noisier, and IgG-1D-noisiest. We report per-image FSC and neighborhood similarity on the reporting split, along with the gaming gap curve over accepted edits.

Two checks are decisive. The first is the gaming gap: the visible-score loop should show rising $S_{\text{vis}}$ with flat or falling $S_{\text{true}}$, and the sealed loop should not. The second is the noise-only test: final programs from each arm are run on pure noise, and any arm that produces a structure correlated with its reference is disqualified.

Transfer checks then apply evolved programs, frozen, to IgG-RL and Spike-MD to test whether the edits generalize across heterogeneity types. They also apply the programs to two CESPED targets under the package's 2-fold gold-standard protocol, to confirm that resolution gains survive real experimental noise.

Finally, we attribute credit. Each accepted edit is reverted one at a time, and we record its contribution to $S_{\text{true}}$. This tells biologists which algorithmic changes mattered, not only that the agent helped.