Re-Fit Before You Trust: Execution-Verified Curation of Legacy Enzyme Kinetics

Modern machine learning models of enzyme behavior are trained on databases of kinetic constants — the maximum reaction speed $V_{max}$, the substrate concentration at half that speed ($K_m$), and the turnover number $k_{cat}$ — that were themselves transcribed from decades of papers. Two recent results suggest this inherited record is quietly broken. First, when a differentiable constraint-based model of E. coli metabolism was allowed to adjust the BRENDA-curated turnover numbers, prediction accuracy against knock-out metabolomic data improved by $30 \pm 3\%$, because historical in vitro values systematically underestimate in vivo catalytic capacity (Interrogating the effect of enzyme kinetics on metabolism using differentiable constraint-based models). Second, an LLM pipeline can now extract over 91,000 kinetic entries from ~3,500 papers with F1 above 0.9 (Enzyme Co-Scientist) — but it copies numbers as strings, never asking whether the reported values are arithmetically consistent, whether they came from an obsolete fitting method, or whether they reproduce the paper's own raw data.

The core idea of this proposal is simple: an extraction pipeline should not merely read kinetic papers, it should re-execute them. Before a legacy value enters a database, the system re-fits the paper's reported raw velocity-versus-substrate data with modern nonlinear regression, checks that the reported $K_m$ and $V_{max}$ actually generate the published curves, checks that $k_{cat}/K_m$ equals the extracted $k_{cat}$ divided by the extracted $K_m$, and records how the original authors fit their data — in particular whether they used the Lineweaver-Burk double-reciprocal linearization, which distorts error structure and biases parameter estimates. This is the automation of a curation discipline that existed only in human form: the BioModels Database curators who re-simulated published kinetic models against published figures and wrote back to authors when the curves did not match (Curation and annotation for BioModels Database) — a practice that was abandoned as a bottleneck precisely when the literature outgrew it.

Core Claim

The claim is threefold. First, a large, measurable fraction of legacy kinetic constants fails execution-based verification — the failures fall into named categories (arithmetic inconsistencies between reported ratios and their factors, order-of-magnitude unit errors, parameters that cannot reproduce the paper's own raw data points, and biases introduced by double-reciprocal fitting), and these categories are detectable by an agent without human hints. Second, correcting these failures with a re-fit against raw data changes the aggregate statistics of the corpus in a consistent direction, providing a mechanistic, provenance-level explanation for the in vitro versus in vivo gap documented by Interrogating the effect of enzyme kinetics on metabolism. Third, models trained on the corrected corpus predict better than models trained on the uncorrected corpus at the same architecture and compute, so the correction has downstream value rather than being bookkeeping.

The proposal deliberately makes enzyme kinetics the mechanism and legacy formalization the lens: the curation agent operates in the kinetic domain, but its design principles — structured provenance, FAIR-compliant schemata, execution-based reproducibility testing — come from the scientific legacy formalization literature. The Digital Ludeme Project showed that historical artefacts can be decomposed into machine-readable structural primitives under a strict relational schema (Data Note); we treat an enzyme kinetics paper the same way, decomposing it into primitives with defined semantics: a substrate, a buffer condition, a set of $(v, [S])$ measurements, a fitting method, and reported constants with units. Schmidhuber's reclamation of algorithmic lineage — showing that modern GAN objectives are restricted cases of 1990s formulations (Generative Adversarial Networks are special cases of Artificial Curiosity (1990)...) — motivates the fitting-method dimension: what a field reports as "the kinetic constant" is not time-invariant, because the estimator used to obtain it changed across decades, and the estimator's bias is part of the value's provenance.

Research Program

Three things get built. The first is a provenance schema: a relational data model in which every kinetic entry carries not just the numeric triplet $(K_m, k_{cat}, k_{cat}/K_m)$ but the assay conditions (pH, temperature, buffer, enzyme concentration), the fitting method class (double-reciprocal linearization, Hanes-Woolf, direct nonlinear regression, unreported), the raw measurement set when recoverable, and a verification status field. This extends the schema of Enzyme Co-Scientist, which captures only the numbers and their substrates, and it operationalizes the open problem both Enzyme Co-Scientist and SAKPE explicitly leave unsolved — that kinetic constants are state-dependent measurements whose assay context is discarded at extraction time.

The second is the verification agent itself. Given a paper, it extracts the reported constants and any tabulated or plotted velocity data, then executes a battery of checks. The arithmetic check tests whether the reported ratio equals the ratio of the reported factors within rounding tolerance. The reproduction check, following the BioModels precedent of re-simulating against published figures (Curation and annotation for BioModels Database), fits $v = V_{max}[S]/(K_m + [S])$ directly to the extracted raw points by nonlinear least squares and measures whether the paper's reported constants fall within the confidence region of the re-fit. The estimator-bias check compares the nonlinear-regression estimate against what double-reciprocal linearization of the same points would have yielded, flagging entries whose reported values match the linearized estimate more closely — evidence the original authors fit by the transformation. The agent emits an Issue Card per failure, adopting the task formalism of DCA-Bench, which showed that dataset auditing becomes tractable for LLM agents when discovery is posed as structured issue-finding against a fixed vocabulary of 18 anomaly categories — though none of its categories cover mathematical inconsistency in scientific data, which is exactly the extension we provide.

The third is the benchmark, KineticAudit: a curated set of legacy papers with raw $(v, [S])$ tables, ground-truth fault labels injected and naturally occurring, spanning the enzyme classes represented in the SAKPE merged corpus (SAKPE) plus the inhibition-mechanism literature (e.g., the non-competitive α-amylase inhibition of Bioactive Profiling and Anti-Hyperglycemic Potential of Berberis nummularia and the mixed-type PARP inhibition of Aminomethylmorpholino Nucleosides...), where the diagnostic geometry of double-reciprocal intersections carries mechanism-level information that current schemas discard. The benchmark inherits the multi-level hint hierarchy ($h_0$ through $h_3$) of DCA-Bench so that discovery rates can be measured as a function of contextual guidance.

Baselines

The do-nothing reference point is the current record itself: accept BRENDA and SABIO-RK values as-is, as SAKPE does when it merges those databases into its training set without re-verification. This is the baseline that matters for the downstream experiment, because any predictive gain from a corrected corpus is measured against models trained on the uncorrected one. The strongest existing competitor is the Enzyme Co-Scientist pipeline: OCR plus multi-LLM extraction plus Pandas normalization, which reports an average F1 above 0.9 on protein enzymes and beat BRENDA's own manual curation on the same literature. It is the strongest competitor because it already solves the extraction half of the problem at near-ceiling accuracy and cost; our system will be run with the identical extraction stack, identical PDF corpus, and identical compute, so the only difference is the added verification layer — a controlled comparison rather than a new pipeline. Beating it means not improving extraction F1 (it is already near ceiling and both systems share the stack) but succeeding on a metric it does not attempt: detecting mathematical and reproduction failures in the extracted corpus, where its expected score is zero by construction. The zero-hint agent baseline is a strong reasoning LLM (following DCA-Bench, where the best such agent, DeepSeek-R1, achieved 29.86% success at $h_0$ and roughly 78% at $h_3$) given the paper and asked to find faults without our execution tooling; beating it means the execution layer — actually running the regression rather than reading about it — lifts discovery success substantially above the tool-free agent at matched model and prompt budget. The key ablation removes exactly our ingredient: the same agent, same corpus, same hints, but with the nonlinear re-fit disabled, so only textual and arithmetic checks remain. If most of the gain survives this ablation, the proposal's central claim — that re-execution matters — is wrong, and we will report it as such.

Why not just…?

Why not just trust BRENDA's manual curation? The evidence against this is direct and quantitative. Enzyme Co-Scientist ran the head-to-head comparison and found the aggregation agent achieved higher F1 than the BRENDA database on identical literature — manual curation is not the gold standard it is assumed to be, it is slower and comparably error-prone. More importantly, Interrogating the effect of enzyme kinetics on metabolism showed that BRENDA's in vitro values are systematically wrong for in vivo prediction, with a $30 \pm 3\%$ accuracy penalty left on the table. The remaining fair version of this objection — that human experts should do the verification — is exactly the BioModels bottleneck, which its own curators identified as unsustainable (Curation and annotation for BioModels Database).

Why not just use RAG over the literature? Because the failure is not a retrieval failure. DCA-Bench found that standard retrieval-augmented generation actively decreased curation accuracy in several settings, because injected generic documentation distracted agents from localized tabular errors. The faults we hunt — a $K_m$ that cannot reproduce its own paper's data points — are not described anywhere in the literature; they must be computed. RAG retrieves text about kinetics; the agent must run regression on kinetics. We will still test this alternative on KineticAudit to confirm the degradation transfers to the biochemical domain.

Why not just re-fit everything with nonlinear regression and skip the agent? This is the most serious alternative and we treat it as such. If every paper's raw data were available as a clean table, a script plus least squares would suffice and no language model would be needed. But raw data in legacy papers lives in scanned figures, prose descriptions, and heterogeneous table layouts; locating and parsing it is exactly the document-understanding problem that Enzyme Co-Scientist showed requires OCR plus multi-LLM reasoning. Moreover, many papers report no raw data at all, and deciding that a value is unverifiable — rather than wrong — is a judgment the pure-refit script cannot make and the agent must. The decisive experiment is staged: on the subset of KineticAudit where tables are machine-readable, we run the pure re-fit script; if it matches the full agent's detection rate there, the agent's value is confined to the (expected, and measured) majority of papers where raw data is not cleanly available, and we will report that partition openly.

Why not just prompt a stronger LLM to check the paper end-to-end? This is the zero-hint agent baseline above, and the existing evidence is pessimistic: the best reasoning agent achieves only 29.86% zero-hint discovery success on general dataset auditing (DCA-Bench). Mathematical consistency in kinetic data is plausibly harder than generic documentation mismatch, not easier. If a frontier model with no tooling matched our execution-based agent, the tooling claim would fail — which is why that comparison is the primary experiment, not an afterthought.

Proposed Experiments

Experiment one measures discovery. On KineticAudit (a few hundred papers across EC classes, with both natural faults found by our own manual audit and controlled injected faults: perturbed ratios, shifted decimals, and synthetic raw-data sets whose reported constants are deliberately inconsistent), we report success rate at each hint level, scored by the DCA-Bench evaluation protocol (its GPT-4o evaluator showed 97.8% alignment with human judgment). The comparison set is the tool-free reasoning agent (reference point: DeepSeek-R1's 29.86% at $h_0$ on DCA-Bench; no reported result found on Lacuna for any agent on biochemical-domain fault discovery) and the extraction-only pipeline (expected fault-detection score: zero, since it does not attempt the task). Success means clearing roughly the gap between 29.86% and the ~78% hinted ceiling without hints, purely from execution.

Experiment two measures the estimator-bias effect. For papers where double-reciprocal linearization of the raw points reproduces the reported constants but nonlinear regression yields different ones, we quantify the systematic difference between linearized and nonlinear estimates across the corpus, and test whether the direction and magnitude of that bias is consistent with the systematic underestimation that motivated the turnover corrections in Interrogating the effect of enzyme kinetics on metabolism. We do not need to re-run their genome-scale optimization: if the fitted correction factors they applied correlate with our identified fault and estimator categories on the overlapping enzyme set, the provenance explanation is supported.

Experiment three measures downstream value. We train the SAKPE architecture unchanged on two corpora: the uncorrected merged set (its published $k_{cat}$ $R^2$ is 0.719, versus 0.468 for DLKcat and 0.648 for UniKP on the same data) and our corrected, provenance-annotated set of equal or smaller size, matching tuning budget. Because a smaller corrected corpus could lose to a larger noisy one purely from data volume, we also run the size-matched control: the uncorrected corpus randomly subsampled to the corrected set's size. The claim is beaten only if the corrected corpus outperforms both the subsampled and, ideally, the full uncorrected corpus — a strict test, and one we expect to be informative even if it fails, since a null result would say that current predictive pipelines are insensitive to precisely the errors this field has spent sixteen years worrying about, itself a finding worth publishing.

The honest risks are these: raw data may be unrecoverable in most legacy papers, shrinking the reproduction check to the arithmetic check alone (we will report the recoverability rate as a first-class result); ground-truth fault labels for natural errors rest on our own expert audit, so we will publish the audit and its disagreements; and the estimator-bias story may not fully explain the in vitro/in vivo gap, since assay conditions unrecorded in legacy papers — buffers, missing activators — contribute causes we cannot measure from the text. The proposal's success does not hinge on closing that gap entirely; it hinges on making the record of what legacy papers actually did, and whether their numbers survive re-execution, a formal, machine-readable property of the databases everyone now trains on.