Identity or Evidence? Auditing Evidence Dependence in Agentic Life-Science Benchmarks

Danai Brilli, Maria GeorgilaarXiv 2026 · arXiv

TL;DR

Are AI agents actually "reasoning" through biological data, or are they just recognizing the name of a drug and recalling its clinical trial outcome from memory? This paper introduces an "identity blinding" audit to show that many life-science benchmarks are inadvertently testing memorization rather than evidence-based reasoning, especially when the provided evidence is weak.

What Problem Does This Paper Address?

As AI agents are increasingly deployed for biological discovery, we need benchmarks to measure their ability to reason over complex evidence (like dossiers, sequences, or research papers). However, these benchmarks often include the names of real-world entities—genes, drugs, or diseases.

The authors identify a "Clever Hans" risk: an agent might achieve a high score not by reasoning over the provided evidence, but by recognizing the entity's name and retrieving the answer from its training data. This paper seeks to decouple identity (who the entity is) from evidence (the data provided about them) to see which one is actually driving the model's performance.

Methods

The researchers developed a protocol to audit benchmarks using several "contrast" conditions:

  • BLIND: The entity names (e.g., "BRCA1") are replaced with opaque, "salted" tokens (e.g., "Gene_A"). This keeps the numeric and scientific evidence intact but hides the entity's identity.
  • DELETE: The evidence block is removed entirely, leaving only the identity.
  • PLACEBO: The evidence for one entity is replaced with evidence from a different, unrelated entity.
  • RECALL: A direct probe where the model is asked for specific database records (like the number of clinical programs for a drug) without being given any evidence at all.
  • Redaction Gradient: On the LitQA2 benchmark, the authors gradually redacted the supporting text (0%, 33%, 66%, 100%) while keeping entity names visible to see how the "blinding gap" grows as evidence disappears.

Datasets

The authors audited several prominent life-science benchmarks:

  • PARROT (ours): A newly constructed clinical-outcome benchmark consisting of 286 target-indication instances, testing whether a drug program will advance past Phase II.
  • LAB-Bench: Specifically the LitQA2 (literature snippets), SeqQA (biological sequences), and DbQA (database lookups) tasks.
  • Biomni: Tasks involving gene screens and GWAS (Genome-Wide Association Study) candidate lists.

Key Insights and Ideas

  • The "Blinding Cost" varies by task: If the evidence is a unique biological sequence, blinding the name doesn't matter because the sequence is the evidence. However, for tasks like GWAS, where the "evidence" is just a list of candidates, performance collapses when the name is hidden.
  • Memorization is a major factor: In clinical benchmarks, the models could accurately "recall" the number of clinical programs for a specific drug/target pair without any evidence provided. This suggests the models are accessing a specific "snapshot" of a database they saw during training.
  • Evidence deletion is a noisy signal: Simply deleting evidence (the "DELETE" condition) is a poor test because it changes the prompt's length, format, and semantic context all at once. Blinding identity is a much cleaner intervention for testing content use.
  • Confidence is deceptive: Models were often just as confident when they were wrong (under the "false identity" condition) as when they were right, showing that "stated confidence" is a poor proxy for accuracy in these agents.
Figure 1: The cost of identity blinding varies with available evidence. Left: conditions within a row differ only in entity mentions; numeric evidence is byte-identical. Right: on 96 LitQA2 questions, the supporting passage is redacted in four steps while entity mentions remain intact. The blinding gap grows with redaction (per-instance slope +0.147, p = 0.004).

Key Results

  • Clinical Benchmark: Accuracy dropped from 0.783 (named) to 0.531 (blinded), which is near the majority-class baseline of 0.521. This indicates the "dossier" provided was doing very little of the heavy lifting compared to the drug's name.
  • LitQA2 Redaction: When 100% of the evidence passage was redacted, the gap between named and blinded performance reached +0.135, proving that the model relies on the name when the text is missing.
  • GWAS Task: This showed the most extreme dependence, with a named accuracy of 0.742 plummeting to 0.082 when blinded (chance is 0.093).
  • Recall Accuracy: The "no-evidence" recall probe achieved 0.818 accuracy, actually outperforming the model when it was given a full dossier.
Figure 2: Recall tracks a database-specific count. Among 218 numeric recall responses, the recalled number of programmes rises with the number recorded in the source table (ρ = 0.654). Medians and interquartile ranges are shown.

Limitations and Future Work

The authors acknowledge that using opaque tokens (like "Gene_A") might inherently make reasoning harder for the model, meaning the "blinding cost" they measured might be an upper bound. Furthermore, the study was limited to two models from the same family (Claude).

The paper concludes that future life-science benchmarks must report identity-blinded results and retrieval telemetry to prove they are actually measuring reasoning. They suggest that the only way to truly test these agents is to use "temporally held-out" data—outcomes that occurred after the model's training cutoff.

Figure 4: Stated confidence is weakly related to accuracy across conditions. Accuracy spans 63 points while mean confidence spans 12. The false-identity condition is particularly over-confident.

Summary updated

Abstract

Agentic life-science benchmarks give an agent evidence about a named entity and score an answer against a recorded outcome. That score tests reasoning over evidence only if the evidence, rather than the name, determines the answer. We examine this distinction by varying identity and evidence independently on a clinical-outcome benchmark and published tasks from Biomni and LAB-Bench. Blinding identity matters little when a prompt contains a decisive sequence or passage. It matters a great deal when the prompt is mostly a name and candidate list (0.742 named versus 0.082 blinded; chance 0.093). Across 96 LAB-Bench LitQA2 questions, the gap increases under one fixed passage-redaction scheme (+0.000 to +0.135; per-instance slope +0.147, p= 0.004). On the clinical benchmark, a no-evidence recall probe matches prediction accuracy and recovers a database-specific programme count. So, we argue that benchmarks intended to measure evidence-dependent reasoning should report identity-blinded results, retrieval telemetry, and positive controls.
BibTeX
@misc{brilli2026identity,
  author = {Danai Brilli and Maria Georgila},
  title = {{Identity or Evidence? Auditing Evidence Dependence in Agentic Life-Science Benchmarks}},
  year = {2026},
  eprint = {2610.36510},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.36510},
}
Figure