From Tool State to Scientific Action: Auditing Decision Provenance in Biomedical Research Agents

Xingyu Yi, Xinya Zhang, Yunshen Zhou, Haoyu BaoarXiv 2026 · arXiv

TL;DR

Scientific AI agents can arrive at the "correct" conclusion while citing entirely different—and often incorrect—evidence due to silent hardware errors. This paper introduces BioAgent-DF, a framework for auditing the "decision provenance" of biomedical agents, demonstrating that ensuring scientific integrity requires tracking the entire evidence path, not just the final verdict.

What Problem Does This Paper Address?

As we move toward "AI Scientists," we are increasingly trusting LLM-based agents to retrieve literature, analyze molecular features, and even direct laboratory experiments. However, these agents run on hardware that isn't perfect. The authors investigate Silent Data Corruption (SDC)—subtle numerical errors occurring on faulty GPUs that don't trigger system crashes (like NaN or Inf errors) but quietly alter computation results.

The core problem is that scientific agents are often "right for the wrong reasons." An agent might correctly label a medical claim as "SUPPORTED" but, due to a hardware glitch in its retrieval tool, cite a paper about microbe size instead of the relevant molecular mechanism. If these agents are directing real-world resources—like choosing which expensive drug compounds to test in a wet lab—these silent errors can lead to wasted effort or missed discoveries, even if the agent’s final report looks plausible.

Methods

The authors developed BioAgent-DF to audit Decision Fidelity (DF). Their approach involves:

  • Tool-State Interventions: They capture "reference" (healthy) and "candidate" (potentially corrupted) states from scientific tools. They then perform "restoration" (replacing corrupted values with healthy ones at specific coordinates) and "transplantation" (the reverse) to see exactly how numerical changes ripple through the agent's logic.
  • Fault Analysis: They categorized faulty GPUs into two classes (size-dependent and expanded-trigger) based on how they produced arithmetic violations during matrix operations.
  • Certified Commitment (Algorithm 3): A novel protocol that holds an agent's actions until independent arithmetic verification is complete. Crucially, if a fault is detected, the system doesn't just replace the bad data; it restores the agent’s execution state to the point before it read the corrupted values, clearing its "visible history."
  • Streaming Verification: To make this practical, they implemented a streaming version of arithmetic checks that significantly reduces the overhead of verifying GPU outputs compared to standard double-execution methods.

Datasets

The study evaluated agents across several complex biomedical tasks:

  • SciFact: A benchmark for scientific claim verification requiring document retrieval, reading, and citation.
  • hERG Cardiac-Safety Assay: A real-world task where agents must allocate a budget to test compounds for potential heart toxicity (blockade of the hERG potassium channel).
  • Additional Benchmarks: The authors also tested the framework on PBMC (single-cell data), NCI (molecular neighbors), and ESOL (solubility) datasets to ensure broad applicability.

Key Insights and Ideas

  • The Verdict is a Poor Proxy for Correctness: An agent can be robust to numerical noise in its final answer while its internal reasoning (the evidence path) completely diverges. Figure 1: The same verdict from different cited evidence. As seen in Figure 1, both branches of the Gemma model return a "SUPPORT" label for a claim about FoxO3a, but the corrupted branch (C) cites a completely irrelevant document about microbe size.

  • Rewinding is Better than Replacing: In the hERG experiments, the authors found that simply replacing a corrupted prediction file after the agent had already read it failed to fix the agent's subsequent decisions. The agent had already "internalized" the bad data. Only by restoring the execution state to a prior checkpoint could the correct decisions be recovered.

  • Sparsity of Consequential Faults: A very small number of corrupted values (global sparsity) can be responsible for 100% of an agent's changed decisions. Identifying these specific "exposed" regions is key to efficient auditing.
  • Prompting Influences Robustness: Interestingly, using "explicit-example" prompts (providing the agent with clear JSON examples) made the agents more sensitive to evidence changes, sometimes preventing verdict changes that occurred under simpler baseline prompts.
Figure 1: The same verdict from different cited evidence. (a) Reference and candidate tool states are supplied to the same agent model on healthy hardware. Claim 431 concerns ROS-mediated FoxO3a activation in neuronal death. Gemma’s reference branch cites a sentence about ROS and autophagy (19922508/3), and its candidate branch cites ROS sensing microbe size (22696649/1). Neither sentence supports the stated FoxO3a mechanism, yet both branches return the benchmark-correct SUPPORT label. Neither citation matches the annotated evidence, 28937856/5. Sentence indices are zero-based and document identifiers are SciFact IDs. (b) Restoration and transplantation select values at their original coordinates. (c) Tool execution completion, evidence-path fidelity, and verdict correctness are evaluated. H1 and H2 repeat the same reference state.

Key Results

  • Decision Recovery: In the hERG assay allocation task, silent numerical faults changed testing decisions for 39 compounds. Using the "State Restoration" method recovered all 39 changed decisions, whereas simply replacing the faulty prediction file recovered none.
  • Repeatable Path Changes: Across SciFact batches, models like Qwen3.5-4B and Gemma 3 4B IT showed repeatable changes in which sentences they cited when exposed to corrupted tool outputs, even when their final verdicts remained stable. Figure 3: Discordance between candidate and reference paths.

  • Efficiency Gains: The streaming implementation of arithmetic checks reduced the total time for forward execution and verification by 42.4% compared to a standard FP64 implementation, making real-time auditing much more feasible for production environments.

Figure 3: All attempts and within-state variability. (a) Each model schedules 27 combinations × eight batches × seven conditions, or 1,512 episodes. (b) Qwen has 208 jointly completed groups and Gemma 152. Between candidate and H1, read, citation, and verdict discordances are 1, 25, and 0 for Qwen and 32, 35, and 0 for Gemma; H2 and full restoration never differ from H1. (c) Claim 507 differs only in the seventh batch; its adjusted read difference is zero. (d) Summaries weight each of 26 Qwen and 20 Gemma combinations equally. CH, HH, and CC are between-state and within-state discordance fractions; δ = CH −(HH + CC)/2, in percentage points (Appendix H.12).

Limitations and Future Work

The authors acknowledge that their study focused on specific model architectures (Qwen, Gemma, GPT-6.1 Sol) and specific classes of GPU hardware faults. While the "Certified Commitment" protocol is powerful, it assumes the availability of a "trusted" hardware source for re-computation.

Future Work includes:

  • Extending this framework to automated laboratories where agents direct physical robotic instruments.
  • Developing "Recovery Agents" that can autonomously decide which specific calculations to verify or which execution states to restore based on a fixed computational budget.
  • Applying these auditing techniques to broader "AI Scientist" benchmarks where evaluation references are hidden from the agent to prevent data leakage.

Summary updated

Abstract

A scientific agent can give the same verdict while reading and citing different evidence. BioAgent-DF tests this separation using paired reference and candidate values saved at the same scientific computation site. It restores or transplants those values at their original coordinates, then runs the agent on healthy hardware. Across eight SciFact batches, Qwen3.5-4B shows repeatable citation changes in three capture-claim combinations, and Gemma 3 4B IT shows repeatable read and citation changes in four. These seven combinations cover two claims. Full restoration recovers the reference evidence sets while verdicts remain unchanged. Verdict changes observed under the baseline prompt did not recur under the explicit-example prompt. Complementary tool and optimizer experiments distinguish recovery of numerical state, recovery of ranked lists, and restoration of an agent's visible history. These results show that an audit should track both the evidence an agent uses and its final answer. In hERG cardiac-safety assay allocation, silent numerical faults changed testing decisions while the fixed-budget shortlist kept the same number of compounds labeled as blockers. After the agent had used faulty values, restoring its earlier execution state and supplying certified scores recovered all 39 changed decisions, whereas replacing the prediction file recovered none; no incorrect action was committed. A streaming implementation of the arithmetic checks reduced the total time for forward execution and verification by 42.4% relative to the original FP64 implementation, with identical flags and recovery outputs.
BibTeX
@misc{yi2026from,
  author = {Xingyu Yi and Xinya Zhang and Yunshen Zhou and Haoyu Bao},
  title = {{From Tool State to Scientific Action: Auditing Decision Provenance in Biomedical Research Agents}},
  year = {2026},
  eprint = {2611.01527},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01527},
}
Figure