PriorityFEC: Deadline-Aware Streaming FEC for KV-Cache Handoff in Edge Multi-Agent Systems

Han Shen, Qifu Tyler Sun, Fanyang Meng, Ye Wang, Yongsheng LiangarXiv 2026 · arXiv

TL;DR

PriorityFEC is a novel transport scheme designed for multi-agent LLM systems at the edge, where context must be handed off between nodes over unreliable links. By prioritizing high-utility KV-cache blocks and using a reinforcement-learning-based adaptive streaming FEC, it ensures that the most critical model state arrives before strict deadlines, significantly boosting task accuracy compared to standard protocols.

What Problem Does This Paper Address?

In collaborative multi-agent LLM systems, tasks are often passed from one agent to another. To avoid the massive computational overhead of "prefilling" (re-processing) the entire conversation history at every step, agents can share their Key-Value (KV) cache—the materialized intermediate states of the model.

However, at the network edge, this handoff faces a major hurdle: the Time-to-First-Token (TTFT) deadline. KV caches can be massive, and edge links (like 5G or IoT connections) are often bandwidth-constrained and prone to bursty packet loss. Traditional reliable protocols like TCP or QUIC rely on retransmissions, which often take too long, causing the receiving agent to miss its deadline. Conversely, unreliable protocols like UDP result in missing data that can break the LLM’s reasoning.

The authors argue that we shouldn't treat every byte of the KV cache as equally important. If we can't send it all in time, we should at least send the parts the model needs most.

Motivation for deadline-constrained KV-cache handoff. (a) Reliable transport waits for loss recovery, delaying TTFT. (b) PriorityFEC prioritizes high-utility blocks to meet deadlines.

Methods

PriorityFEC tackles this through a two-pronged approach:

  1. Utility-Aware Source Ordering: The authors first profile LLMs (Qwen and Llama) to see which parts of the KV cache are most sensitive to data loss. They found that early layers and the very beginning/end of the token sequence are disproportionately important for model fidelity. PriorityFEC maps these "importance scores" to transport blocks and places high-utility blocks at the front of the transmission queue.
  2. Adaptive Streaming Forward Erasure Correction (FEC): Instead of waiting for retransmissions, FEC sends "repair packets" alongside data. PriorityFEC uses an ACK-sliding Streaming FEC mechanism. Crucially, it employs a Double DQN (Deep Q-Network) scheduler to decide when to send a repair packet versus a new data packet. This RL agent considers the remaining time, the remaining repair budget, and the utility of the blocks currently at risk of being lost.
Fig. 2. Characterization of non-uniform KV-cache utility. (a) Normalized sen- sitivity across layer groups. (b) Normalized sensitivity across relative token- position bins. (c) Teacher-forced top-1 agreement under partial KV retention on held-out prompts. Profile-ranked retention consistently outperforms random retention; error bars denote 95% confidence intervals.

Datasets

The system was evaluated using two representative open-source LLMs:

  • Qwen2.5-7B-Instruct
  • Llama-3.1-8B-Instruct

Performance was tested on three complex question-answering workloads from the LongBench-E benchmark:

  • 2WikiMultiHopQA
  • HotpotQA
  • TriviaQA

Network conditions were simulated using both a synthetic Gilbert-Elliott burst-loss model and real-world 5G SA (Standalone) uplink traces.

Key Insights and Ideas

  • The "Aha" Moment on Utility: The paper proves that equal amounts of recovered KV data can result in wildly different model accuracy. For example, at 40% data retention, picking the "right" blocks (profile-ranked) yields 55.2% agreement with the original model, compared to only 44% for random selection.
  • Opportunity Cost of Repair: In a deadline-constrained environment, every repair packet you send is a transmission slot you didn't use to send a new piece of the KV cache. The RL scheduler learns this trade-off, only "spending" its repair budget when the lost data is high-utility and the deadline is looming.
  • Layer-wise Heterogeneity: LLMs aren't monolithic. The sensitivity analysis (Figure 2a) shows that early layers are often much more critical than middle layers, a finding that directly informs the priority queue.

Key Results

PriorityFEC demonstrates its greatest strength under "tight" deadlines (where there isn't enough time to send the full cache even without losses):

  • Utility Retention: Under a tight deadline ($\rho = 0.8$), PriorityFEC preserved ~99% of the normalized KV utility, while the best non-priority baseline only managed 75-89%.
  • Task Accuracy (F1 Score): On the TriviaQA dataset with Llama-3.1-8B, PriorityFEC achieved a token-level F1 score of 75.2%. In contrast, standard adaptive FEC (FLEC-RT) plummeted to just 16.8% because it didn't prioritize the most important blocks.
  • Computational Feasibility: The RL scheduler is extremely lightweight. The decision-making process takes only ~77 microseconds, which is well within the ~112-microsecond window required to keep up with a 100-Mbps link.

Limitations and Future Work

  • Scalability: The current profiling method requires constructing full attention tensors, which might become a bottleneck for extremely long contexts (e.g., 32k or 128k tokens).
  • Generalization: While the system was tested on 5G traces, the authors note that more extensive testing across different RTTs, bandwidths, and non-stationary network conditions is needed to ensure broad generalization.
  • Attention Masking: While the paper mentions using an explicit partial-KV attention mask, the primary results rely on "zero-filling" missing data. Further exploration into how different recovery strategies interact with the transport layer could be fruitful.

Summary updated

Abstract

Multi-agent large language model (LLM) systems are increasingly deployed at the edge to support complex tasks that require collaboration and information exchange among multiple agents. However, when task contexts are passed across agents, shared context that has already been processed by upstream agents often needs to be prefilled again, resulting in substantial redundant computation. Reusing an already prefilled key--value (KV) cache can avoid this recomputation, but requires the reusable KV state to be transferred across the network before the receiving agent can begin generation. Under strict time-to-first-token (TTFT) deadlines and unreliable edge links, delivering the complete KV cache may be infeasible, making it necessary to prioritize and protect the most valuable KV information within a limited transmission budget. This paper proposes PriorityFEC, a utility-aware KV-cache handoff scheme that jointly exploits model-side KV importance and transport-side reliability. PriorityFEC first constructs a utility-aware source stream by placing high-utility KV blocks at earlier transmission positions. It then protects the ordered stream using a feedback-adaptive Streaming Forward Erasure Correction (FEC) mechanism, which schedules repair packets according to receiver feedback before the TTFT deadline. We implement PriorityFEC on a distributed edge LLM serving prototype and evaluate it with Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct across multiple datasets. Results show that PriorityFEC achieves superior task quality over the compared transport schemes, with its advantages becoming particularly pronounced under stringent deadline constraints. These results demonstrate the importance of jointly optimizing KV-cache utility and deadline-aware transport reliability when reusable LLM state must be handed off across unreliable edge networks.
BibTeX
@misc{shen2026priorityfec,
  author = {Han Shen and Qifu Tyler Sun and Fanyang Meng and Ye Wang and Yongsheng Liang},
  title = {{PriorityFEC: Deadline-Aware Streaming FEC for KV-Cache Handoff in Edge Multi-Agent Systems}},
  year = {2026},
  eprint = {2610.21390},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.21390},
}
Figure