BIO-MEMART: Biometric-Aware KV Cache Memory for Multi-User LLM Agents

Yanhong Qian, Xuanying He, Qingguo Meng, Shihao Ding, Xingbo Dong, Zhe JinarXiv 2026 · arXiv

TL;DR

Bio-MemArt introduces a biometric security layer to Large Language Model (LLM) KV-cache memory, ensuring that shared agents only retrieve a user's private history after physical identity verification. By filtering the memory pool with face or palmprint data before semantic retrieval, it prevents cross-user privacy leaks while maintaining the massive efficiency gains of KV-cache reuse.

What Problem Does This Paper Address?

As LLM agents transition from simple chatbots to persistent personal assistants, they need long-term memory. The most efficient way to implement this is through KV (Key-Value) cache reuse. Instead of re-processing a massive text history every time you ask a question, the system stores the model's internal mathematical states (KV blocks) and reloads them directly.

However, a major security gap emerges in shared environments like smart homes or public terminals. If multiple users share one agent, their KV blocks sit in a collective pool. Standard retrieval methods look for semantic relevance—they find the memory that best matches the topic of the query. The problem? Semantic relevance doesn't imply authorization. If User A asks about "my medical history," a shared agent might accidentally retrieve User B's private data simply because it is the most relevant "medical history" in the cache.

Methods

The authors propose Bio-MemArt, which extends the existing MemArt framework by adding a physical-user identity gate.

  1. Biometric Tagging: Every stored KV memory block is augmented with a normalized biometric template (e.g., a face or palmprint embedding).
  2. Identity-First Retrieval: The retrieval process is split into two distinct stages:
  3. Biometric Filtering: When a user issues a query, they provide a biometric "probe" (like a quick face scan). The system compares this probe against all stored templates using cosine similarity. Only blocks that exceed a statistically determined threshold $\tau$ enter the "authorized candidate pool."
    • Semantic Retrieval: The system then performs standard latent-space retrieval only within that authorized pool to find the most relevant memories for the specific question.
  4. Threshold Estimation: Rather than picking a random number, the authors estimate a benchmark-specific threshold by analyzing the overlap between "matched" and "mismatched" similarity distributions, ensuring a rigorous balance between security and accessibility.
Figure 1: Overall pipeline of Bio-MemArt. (a) Biometric identity verification matches the current user’s biometric feature against enrolled identities and activates only the corresponding owner memory pool; unmatched requests receive no authorized memory. (b) Personalized KV-memory retrieval is then performed inside the authorized candidate pool: the request is encoded into query KV states, compressed keys are used for top-K block selection, selected memories are merged under a unified position layout, and the decoder reuses the retrieved KV cache to generate the final answer.

Datasets

The researchers evaluated Bio-MemArt using a combination of dialogue and biometric datasets:

  • Dialogue QA: They used LoCoMo, a benchmark for long-term conversational memory that tests Multi-Hop, Temporal, and Single-Hop reasoning.
  • Face Recognition: Seven public benchmarks including AgeDB-30 (age variation), LFW (frontal views), and CPLFW (pose variation).
  • Palmprint Recognition: Ten protocols from five datasets, including CasiaM, IITD, and PolyU, covering various light wavelengths and contact methods.

Key Insights and Ideas

  • Privacy as a Hard Constraint: By filtering before semantic search, Bio-MemArt ensures that unauthorized data is never even "seen" by the retrieval algorithm. This is more robust than trying to teach an LLM to "ignore" certain parts of its context.
  • Preserving Efficiency: The core appeal of KV-cache memory is avoiding the "prefill" cost of processing long texts. Bio-MemArt proves that adding a security layer doesn't destroy these gains. The biometric matching is computationally negligible compared to LLM inference.
  • Behavioral Safety: As shown in Figure 2, when an owner asks a question, they get a personalized answer. When a non-owner asks the same question, the system correctly fails to find the memory and provides a safe, non-committal response instead of leaking data.
Figure 2: Visualization of the response process under owner and non-owner queries. Given the same question, the owner branch passes identity authentication, retrieves only owner memories, and returns the correct personalized answer about the Boston restaurant. The non-owner branch fails identity matching, cannot access that user’s private memory pool, and returns a safe non-committal response instead of leaking personalized content.

Key Results

  • Security Success: Across face benchmarks, the system achieved an average owner success rate of 95.71% while maintaining a non-owner access rate of only 0.86%. Palmprint results were even stronger, with a 97.60% owner success rate.
  • Massive Speedup: In efficiency tests, "Full-context prompting" (re-reading everything) required an average of 18,781.96 prefill tokens. Bio-MemArt slashed this to just 28.57 tokens, a massive reduction in computational overhead.
  • QA Performance: The "Avg. Gap" in QA scores (F1/BLEU) between owners and non-owners was significant (e.g., ~18.23 points for LFW), confirming that the biometric gate effectively isolates personal knowledge.

Limitations and Future Work

  • Biometric Robustness: The study uses pre-extracted embeddings. In real-world deployments, factors like poor lighting, motion blur, or adversarial "spoofing" (e.g., using a photo of the owner) would need to be addressed.
  • User Scaling: The experiments focused on pools of up to five users. Future work could explore how the system handles much larger, enterprise-scale memory pools.
  • Template Updates: The current model uses static biometric templates. A more advanced system might need to update templates over time (e.g., as a user ages) without requiring a full re-indexing of the KV cache.

Summary updated

Abstract

KV cache is evolving from a serving optimization into an external memory substrate for long-term LLM agents. In a shared multi-user deployment, however, reusable KV blocks introduce a missing access-control question: semantic relevance alone cannot determine whether a memory block is authorized for the current physical user. We propose Bio-MemArt, a biometric-aware KV-cache memory framework for multi-user LLM agents. Bio-MemArt attaches a normalized biometric template to each stored KV memory block, filters the shared memory pool with the current user's biometric probe, and then runs the original MemArt retrieval and KV reuse pipeline only inside the authorized candidate pool. This design preserves latent-space retrieval, direct cache reuse, and decoupled position encoding while adding physical-user access control to shared KV memory. We evaluate Bio-MemArt under Owner and Non-owner query conditions on long-term dialogue QA with face and palmprint benchmarks. Across face benchmarks, the average owner and non-owner biometric success rates are 95.71% and 0.86%; across palmprint benchmarks, they are 97.60% and 2.00%. In the efficiency study, average prefill tokens drop from 18,781.96 under full-context prompting to 28.57 with Bio-MemArt, showing that biometric gating preserves the low-token operating regime of KV-cache memory.
BibTeX
@misc{qian2026bio,
  author = {Yanhong Qian and Xuanying He and Qingguo Meng and Shihao Ding and Xingbo Dong and Zhe Jin},
  title = {{BIO-MEMART: Biometric-Aware KV Cache Memory for Multi-User LLM Agents}},
  year = {2026},
  eprint = {2609.08566},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.08566},
}
Figure