BioMem: Agentic Memory for Biomedical LLM Agents

Mingchen Li, Yongkang Xiao, Rumeng Li, Rui Zhang, Hong YuarXiv 2026 · arXiv

TL;DR

BioMem presents the first systematic investigation into "agentic memory"—persistent, evolving information systems—for biomedical LLMs. The authors identify critical failure modes in how models retrieve and utilize medical data and introduce HMRU (Hybrid Memory Retrieval and Utilization), a training-free framework that improves biomedical reasoning by 8.6% and closes nearly half the performance gap between memory-augmented systems and full-context models.

What Problem Does This Paper Address?

While Retrieval-Augmented Generation (RAG) has become a standard for grounding LLMs in external knowledge, biomedical applications require more than just static lookups. Agents in this domain must handle agentic memory: the ability to store, update, and reuse information across long-term patient interactions and multi-session clinical reasoning.

However, we don't yet understand the reliability of these persistent memory systems in a high-stakes medical context. This paper addresses three core questions:

  1. Fidelity: Does the memory accurately reflect the original source after multiple updates?
  2. Retrieval vs. Utilization: If a model retrieves the correct "answer-bearing" memory, does it actually use it to produce the right answer?
  3. Task Sensitivity: How do factors like retrieval depth and class imbalance affect clinical predictions compared to simple knowledge retrieval?

Methods

The researchers first benchmarked five existing memory strategies (including A-MEM, Mem0, and LightMem) across four state-of-the-art LLMs. Finding significant gaps in how these models handled biomedical nuances, they proposed HMRU (Hybrid Memory Retrieval and Utilization).

HMRU is a training-free framework consisting of three main components:

  • Hybrid Retrieval: Combines dense vector retrieval (semantic) with BM25 (lexical) to ensure that specific medical terminology isn't missed.
  • Cross-Encoder Reranking: Refines the retrieved set to prioritize the most relevant evidence.
  • Task-Aware Utilization: Uses specialized prompt instructions to guide the model on how to weigh retrieved evidence, emphasizing entity-relation consistency for reasoning and outcome prevalence for clinical predictions.

Datasets

The study evaluates memory performance across three distinct settings:

  • Biomedical Knowledge Reasoning: Using RiTeK-PharmKG and RiTeK-ADInt, which require reasoning over complex medical knowledge graphs.
  • Clinical Prediction: Utilizing MIMIC-III (Mortality and Readmission), TCGA-Reports (Cancer classification), and MIMIC-IV OOD (Opioid overdose prediction).
  • Long-term Conversational Memory: Using the medical subset of RealMem, which tests temporal reasoning and information updates across multiple sessions.

Key Insights and Ideas

  • The Utilization Gap: Simply retrieving the right information is not enough. The authors found that even when "answer-bearing" memories were present in the prompt, models often failed to utilize them correctly. Evidence-to-outcome flows showing that retrieving relevant evidence does not guarantee correct utilization.

  • Memory Drift: As agentic memory systems "evolve" (summarizing and rewriting notes over time), they tend to introduce source-unsupported information. In some tasks, up to 44% of memory nodes contained "hallucinated" or unsupported context after several rewrites.

  • Similarity $\neq$ Outcome: In clinical prediction, retrieving a "semantically similar" patient record doesn't necessarily help predict a future outcome. This highlights a fundamental limitation of standard RAG for prospective risk assessment.
  • History Length is a Red Herring: Contrary to expectations, longer conversation history didn't consistently degrade performance. Instead, the accessibility and integration of specific evidence were the primary bottlenecks.
Evidence-to-outcome flows showing that retrieving relevant evidence does not guarantee correct utilization.

Key Results

HMRU demonstrated significant improvements over baseline agentic memory systems:

  • Overall Gains: Achieved an average relative improvement of 8.6% across QA and conversational benchmarks.
  • Error Reduction: Reduced conversational answer errors by 24.9%.
  • Closing the Gap: Closed 48.0% of the performance gap between standard memory systems and "Full-Context" inference (where the model sees the entire history at once).
  • Mechanistic Success: The "Task-Aware" component was particularly effective at reducing errors involving multiple entities and inappropriate "yes/no" responses. Mechanistic analysis showing how HMRU components reduce retrieval and utilization failures.
Mechanistic analysis showing how HMRU components reduce retrieval and utilization failures.

Limitations and Future Work

The authors note that while HMRU excels at knowledge retrieval and reasoning, clinical outcome prediction remains a significant challenge. Standard retrieval objectives (finding similar text) are often insufficient for predicting risks like opioid overdose.

Future work should focus on outcome-aware retrieval—systems that don't just look for similar patients, but specifically look for patients with similar trajectories and outcomes. Additionally, addressing the "fidelity drift" in evolving memories is crucial for maintaining the long-term safety of biomedical agents.

Summary updated

Abstract

Large language models (LLMs) have demonstrated considerable potential in biomedical applications, while agentic memory systems have emerged as a promising approach to preserving and utilizing information across tasks and interactions. However, a fundamental question remains: how effectively and reliably do these systems retain, retrieve, update, and utilize biomedical information? In this study, we systematically investigate agentic memory in the biomedical domain, focusing on task effectiveness, retrieval quality, information fidelity, and long-term robustness. We evaluate five memory approaches across four LLMs in three representative settings: biomedical knowledge reasoning, clinical prediction, and long-term conversational memory. Our analyses reveal how memory construction, evidence relevance, and conversational history shape downstream biomedical performance, highlighting the distinct challenges of retrieving relevant information, preserving its fidelity, and effectively utilizing it for reasoning. Building on these findings, we introduce Hybrid Memory Retrieval and Utilization (HMRU), a training-free framework that jointly enhances evidence accessibility and task-aware utilization to improve biomedical reasoning and long-term conversational memory. HMRU achieves an average relative improvement of 8.6% across biomedical question-answering and conversational memory benchmarks, with additional gains in selected clinical prediction tasks. It further reduces conversational answer errors by 24.9% and closes 48.0% of the performance gap to full-context inference. Our findings provide a systematic characterization of biomedical agentic memory and practical insights into developing more effective and reliable memory-augmented LLM systems.
BibTeX
@misc{li2026biomem,
  author = {Mingchen Li and Yongkang Xiao and Rumeng Li and Rui Zhang and Hong Yu},
  title = {{BioMem: Agentic Memory for Biomedical LLM Agents}},
  year = {2026},
  eprint = {2611.13426},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13426},
}
Figure