Do Thermodynamic Loops Corrupt Flux-Sampling Predictors of Gene Essentiality?

Computer models of cell metabolism are often scored only against other computer models. The best recent methods for predicting which genes a cell cannot live without share one recipe. They draw thousands of random "flux distributions" (feasible patterns of reaction rates, meaning how fast each chemical reaction runs) from a genome-scale metabolic model, then train a classifier on those samples. The classifier's predictions are then checked against real knockout screens. These samplers enforce only mass balance: what enters each metabolite pool must leave it. They do not enforce the second law of thermodynamics, so their samples contain futile cycles, closed loops of reactions that circulate flux with no energy input, the metabolic version of a perpetual motion machine. Separately, a line of work has built samplers that remove these loops exactly, but those samplers have been judged only on in-silico mixing diagnostics, never on whether they predict anything real.

This note proposes a controlled study at that seam. We take the published sampling-plus-learning pipelines, swap only the sampler across a ladder of increasing thermodynamic rigor, and score each variant against the same real CRISPR and deletion screens. The main new element is a loop-stratified evaluation. We split genes by whether their reactions can take part in a futile cycle, and we ask whether any benefit of loop removal falls on exactly those genes. The design also asks whether a cheap per-sample loop fix captures most of the benefit of exact loopless sampling. If it does, the practical answer is "use the cheap fix," and the proposal is built so that answer can win.

Core Claim

We claim that loop contamination in sampled fluxes is a localized error. It should hurt essentiality predictions mainly for genes whose reactions belong to loop-capable subnetworks. It should leave the rest of the genome nearly untouched, which would explain why loop-contaminated pipelines still score well in aggregate. If this is right, the aggregate metrics reported so far hide a systematic, fixable failure. The fix should show up as a large gain on the loop-capable stratum and near-zero change elsewhere. If loop removal changes nothing even on that stratum, the field can stop worrying about loops in sampling-based predictors, which is also worth knowing.

Grounding in Prior Work

Two papers motivate the question directly. FluxGAT samples the Chinese hamster ovary (CHO) cell model iCHO2291 with artificial-centering hit-and-run (a standard random-walk sampler over the mass-balance polytope). It builds a reaction graph weighted by sampled mass flow and trains a graph attention network. Validated on a CHO CRISPR screen of 15,028 genes, it reaches PR-AUC 0.765 and sensitivity 0.576, against 0.304 for plain FBA. The authors concede that their sampler produces thermodynamically infeasible loops that "could affect performance," but they never test this. Flux Cone Learning samples the flux cone of each knockout mutant and trains random forests on the samples. It reports 95% accuracy on E. coli essentiality with iML1515, 6 points better than FBA on essential genes, and it notes that a few boundary samples suffice. Our proposal takes the learning architectures of both papers unchanged. What we change is the sampler, and what we add is the stratified analysis. Boundary samples are where unbounded cycles live, so Flux Cone Learning is the pipeline where we expect loops to matter most.

Two papers supply the thermodynamic machinery, and neither has been checked against real data. LooplessFluxSampler introduces adaptive directions sampling on a box (ADSB), which samples uniformly over the non-convex loopless flux space. It reports convergence near psrf 1.0, where psrf is the potential scale reduction factor used to check that independent sampling chains agree, and speedups up to 1000-fold on yeast iMM904. Its evaluation is entirely in silico. It also cannot sample reactions that carry flux only through loops, and that limitation becomes part of our analysis. Troppens et al. solve loopless flux balance analysis exactly as a mixed-integer program using combinatorial Benders' decomposition. That approach solves 93% of BiGG instances in 58 seconds on average but only 1 of 11 large yeast models. It is scored on solver metrics alone, and it bounds how far we can push exact per-knockout loopless optimization.

Two further papers constrain the design. Dreyfuss et al. use limed-FBA, a linear-programming penalty on metabolite dilution, to remove cycles in Neurospora crassa. They reach 93.4% sensitivity and 92.9% specificity on 303 held-out phenotypes, and they fix a specific error in which an internal cycle made arg-14 look non-essential. That single anecdote is the best existing evidence that loops cause real prediction errors, and it shows a cheap linear method can suffice. Edirisinghe et al. show that automated reconstructions leak energy through spurious electron transport loops. This warns us that some "loops" are annotation errors rather than sampler artifacts. We therefore run every variant on the same curated models, so that errors in the model are held fixed and only the sampler varies.

Research Program

We first compute, for each model, the loop-capable reaction set. These are the internal reactions that can carry nonzero flux when all exchange reactions with the environment are closed, found by flux variability analysis. Genes are then assigned by their gene–reaction rules to a loop-capable stratum or a loop-free stratum. The stratum sizes are themselves a first result, since nobody has reported what fraction of experimentally screened essential genes touch loops.

Next we generate four sample sets per wild-type and knockout model, all with the same number of samples:

  • Raw. Hit-and-run samples as used in the published pipelines.
  • Projected. The same samples, each passed through a linear program that keeps its exchange fluxes fixed and minimizes total internal flux. This deletes cycle flux one sample at a time.
  • Dilution-penalized. Samples drawn under limed-FBA-style dilution constraints.
  • Exact loopless. ADSB samples, uniform over the loopless space.

For each sample we record a loop fraction: the share of its total absolute flux removed by the projection. The loop fraction gives a continuous measure of contamination, which we can relate to per-gene prediction error.

Each sample set is fed to the two fixed learners: Flux Cone Learning's random forest and FluxGAT's graph attention network. Hyperparameters are tuned once on raw samples and then frozen, so that later variants cannot gain from extra tuning. Reactions that ADSB cannot sample because they require loops receive zero flux. We report their genes separately, since they are a set where exact soundness may make predictions worse. Finally we check whether the in-silico sampler diagnostics (psrf, effective sample size) predict the real-screen gains. That check speaks directly to how far simulation-only evaluation can be trusted.

Baselines

The naive reference is plain FBA essentiality: knock out the gene and check whether maximal biomass drops below a threshold. The file reports sensitivity 0.304 and specificity 0.985 for it on the CHO screen in the FluxGAT paper. Any sampling variant that fails to beat it on the loop-capable stratum is uninformative.

The strongest existing methods are the published pipelines run with raw samples. For CHO these are FluxGAT, which reports PR-AUC 0.765, sensitivity 0.576 and specificity 0.977 (FluxGAT). For E. coli iML1515 it is Flux Cone Learning, which reports 95% accuracy (Flux Cone Learning). For Yeast9 we found no reported per-metric result on Lacuna for Flux Cone Learning, so we reproduce it ourselves. We re-run these pipelines ourselves on the same splits rather than quoting their numbers against ours. For N. crassa iJDZ836, the limed-FBA result of 93.4% sensitivity and 92.9% specificity on 303 phenotypes (Dreyfuss et al.) is the reference that any sampling variant must match.

Beating a baseline means a gain in PR-AUC on the loop-capable stratum whose bootstrap interval over genes excludes zero, together with no significant loss on the loop-free stratum. We report aggregate metrics too, but a small aggregate gain alone would not count as success.

The key ablation is feature masking. Raw samples are kept, but all features on loop-capable reactions are zeroed or removed before training. This separates two stories. In one, loops hurt because they contaminate features on a few reactions. In the other, they hurt by distorting the global geometry the classifier reads. If masking matches ADSB, the geometry story is wrong.

Why not just…?

Why not just remove loops from each sample after the fact instead of running an exact loopless sampler? This is the most serious threat, and it is included as the projected arm rather than argued away. The per-sample projection removes all cycle flux at the cost of one linear program per sample. It does not produce a uniform distribution over the loopless space, but the classifiers may not care. If projection matches ADSB on the loop-capable stratum, we will recommend it and treat uniform loopless sampling as unnecessary for this task. The 1000-fold speedups reported for ADSB would then matter for sampling as such, not for predicting phenotypes.

Why not just use a linear dilution penalty as FARM does? limed-FBA has the only direct real-world evidence here, the arg-14 correction, and strong held-out accuracy in N. crassa (Dreyfuss et al.). It is therefore a full arm, not a straw man. Its weakness is that the penalty strength is a free parameter. We sweep that parameter on the training split only. If the dilution arm ties with the exact arms, that settles the question the file raises: whether intractable exact formulations buy anything biologically.

Why not just drop the loop reactions from the features? That is the masking ablation above. It is cheaper than either sampling fix and would be the preferred recommendation if it performs equally well. We expect it to fall short only if loop flux in the raw samples also inflates flux through reactions that are not themselves in a loop. The loop-fraction versus error analysis tests that directly.

Why not skip metabolic models and learn essentiality from sequence or expression? FluxGAT already reports beating support vector, multilayer perceptron, random forest and topology-only graph baselines on the CHO screen. That answers whether flux samples carry signal. It does not answer whether their thermodynamic defects matter, which is our question, so we do not repeat this comparison.

Decisive Checks

The study runs on E. coli iML1515 and smaller E. coli models, Yeast9, iCHO2291 with the CHO CRISPR screen, and iJDZ836 with its 303 held-out phenotypes. Splits are made by gene and fixed across all arms.

The central hypothesis is confirmed if two things hold on at least two of the organisms. First, some loop-removing arm improves the loop-capable stratum. Second, per-gene improvement correlates with the loop fraction of that gene's knockout samples. The hypothesis is refuted if gains are either absent or spread evenly across strata. An even spread would mean loop removal acts as generic regularization rather than correcting physically wrong predictions.

A second check targets the genes whose reactions ADSB cannot sample. If exact soundness makes predictions worse there, that would be the first evidence that the full-support limitation has a real biological cost.

A third check asks whether psrf and sampling speed rank the arms the same way real-screen accuracy does. If they do not, then in-silico sampler benchmarks, the only evaluation loopless samplers have so far received, are the wrong yardstick for this use.

The main compute risk is ADSB on iCHO2291, which is roughly seven times larger than iMM904. If ADSB does not converge there, we report CHO with the projection and dilution arms only and say so plainly.