Question-answering systems that must chain facts, such as "find the director of Inception, then find where he was born," work much better when they retrieve in several rounds. Each round's partial reasoning becomes the next search query. Every leading method for this "multi-hop" problem assumes the language model and the document index sit in the same data center, where a round of retrieval costs only compute. That assumption breaks for a vehicle, robot, or handheld device that runs its own reasoning model but queries a shared index over a weak, crowded wireless link. There, every hop costs a radio round trip, and the radio packets are short enough that the classical capacity formula no longer predicts how often they fail.
This note proposes to treat each retrieval hop as a short-packet transmission and to design both the query and its radio budget together. Concretely, the device sends a few-dozen-byte quantized vector, taken from the model's middle layers, instead of a text sub-query. It then splits a fixed deadline and energy budget across hops, query sizes, and returned-evidence sizes using finite-blocklength error formulas instead of Shannon capacity. The claim to test is specific. On low-SNR, narrowband links, this co-design should answer more MuSiQue and 2WikiMultihopQA questions correctly within a deadline than either text-query iterative retrieval or one-shot retrieval. Most of the gain should come from two sources: shrinking the query packet, and balancing three failure sources (radio error, retrieval miss, and deadline violation) to the same order of magnitude.
Core Claim
Finite-blocklength information theory describes how reliability, rate, and packet length trade off when a packet is only tens to hundreds of channel uses long. Its central result is the normal approximation used in Joint Sum Rate and Error Probability Optimization: Finite Blocklength Analysis and Finite-Blocklength Information Theory. It says that sending $b$ information bits in $n$ channel uses with error probability $\epsilon$ requires approximately
$$b \approx nC - \sqrt{nV}\,Q^{-1}(\epsilon),$$
where $C$ is the channel capacity per use, $V$ is the channel dispersion (how much the channel's quality fluctuates), and $Q^{-1}$ is the inverse Gaussian tail function. The square-root penalty is largest exactly when packets are short. That makes a 200-byte text sub-query and a 24-byte vector code differ in cost by more than their size ratio once reliability targets are tight.
The multi-hop side contributes the observation that a retrieval query need not be text. Optimizing Multi-Hop Document Retrieval Through Intermediate Representations, called L-RAG, maps middle-layer hidden states directly into a dense retriever's embedding space through a small learned network. This lets it fetch the second-hop document without decoding a text query. L-RAG was built to save local compute. We repurpose it to save radio resources. We quantize that projected vector to $b$ bits and choose $b$ per hop.
The design problem then becomes an allocation across hops $h = 1,\dots,H$. We pick the query size $b_h$, the number of returned evidence sentences $k_h$, and the channel uses for each uplink and downlink packet. The goal is to minimize end-to-end failure:
$$\epsilon_{\text{total}} \approx \sum_{h=1}^{H}\left(\epsilon^{c}_{h} + \epsilon^{r}_{h}(b_h,k_h)\right) + \epsilon^{d},$$
subject to total channel uses and energy within budget. Here $\epsilon^c_h$ is the packet error at hop $h$ given by the formula above, $\epsilon^r_h$ is the probability that retrieval misses the needed evidence (measured empirically as a function of query bits and evidence count), and $\epsilon^d$ is the probability of missing the deadline. The sum is a union bound on the multiplicative cascade, and it is tight when each term is small.
Cross-layer Transmission Design for Tactile Internet showed for robotic control traffic that the most power-efficient design balances coding error, queueing violation, and proactive dropping at the same order of magnitude. Our hypothesis is that the same balance rule holds when one of the three components is a retrieval miss rather than a network event. If it does, there is a simple design rule for anyone deploying agentic retrieval at the edge.
What Is New Relative to Prior Work
The closest work falls into three groups. The first group established the gap and supplies our baselines. Interleaving Retrieval with Chain-of-Thought Reasoning (IRCoT) uses each reasoning sentence as the next query. Chain-of-Retrieval Augmented Generation (CoRAG) fine-tunes a model to write sub-queries and scales test-time search. DRAGIN triggers retrieval from token uncertainty and attention. All three send text queries and treat a retrieval round as free apart from compute. We price each round in channel uses and failure probability.
The second group supplies methods we reuse. L-RAG provides the hidden-state query. PAGE-RAG provides minimal sufficient selection of supporting facts under a token budget, which we reuse server-side to shrink the downlink payload. PAGE-RAG's budget is a count of context tokens. Ours is a count of channel uses with an error probability attached, and the two are not interchangeable: halving bytes does not halve channel uses at fixed reliability.
The third group constrains the design. Energy-Efficient Packet Scheduling With Finite Blocklength Codes shows that Shannon-rate models underestimate loss and energy for short packets, which motivates our finite-blocklength allocation. Saddlepoint Approximations for Short-Packet Wireless Communications warns that the normal approximation can be loose under fading. We therefore validate allocations against saddlepoint estimates. Alleviating Distortion Accumulation in Multi-Hop Semantic Communication studies distortion accumulating over physical relays, not over reasoning steps. Its multi-hop is a different object from ours, though its finding that errors compound motivates per-hop allocation.
We do not adopt the more speculative mappings in the research file, such as treating a context window as a codeword. The proposal uses finite-blocklength theory only for what it literally describes, which is radio packets.
Research Program
The first stage is a feasibility measurement. It determines whether the radio is ever a meaningful share of multi-hop cost, and it could end the project. We will profile per-hop LLM latency for Llama-3.1-8B-Instruct, the model CoRAG fine-tunes, on edge-class hardware, both with and without L-RAG's decoding shortcut. We then compute radio time and energy for text and vector queries across SNR from −5 to 15 dB and per-device resource grants from roughly 100 to 2,000 channel uses per mini-slot. These ranges cover cell-edge vehicular and many-user narrowband cases. If radio cost stays under about 10% of end-to-end time and energy in every plausible regime, we report that negative result and stop. We expect the regime to exist mainly at low SNR with small grants, especially once L-RAG removes decoding latency.
The second stage builds the quantized hidden-state query. We train L-RAG's projection into Contriever space as in the original work. We then add product quantization of the projected vector at 8 to 64 bytes, fine-tuned end-to-end against hop-level retrieval recall. This yields measured curves $\epsilon^r_h(b,k)$ for each hop position on MuSiQue, 2WikiMultihopQA, and HotpotQA in the open-corpus setting.
The third stage is the allocator. Using the measured retrieval curves and the finite-blocklength formula, we search jointly over hop count, per-hop bits, per-hop evidence count, and channel uses. The search space is small enough for exhaustive search per deadline. The fourth stage tests whether the optimum obeys a balance rule and whether a closed-form "equalize the three failure terms" heuristic matches exhaustive search.
Baselines
The naive reference point is one-shot retrieval. The device sends the raw question as text once, receives the top-$k$ passages, and answers. It costs one round trip. CoRAG reports its weakness on the hardest benchmark: retrieval recall on MuSiQue rises from 29.0% single-step to 47.1% with chain-of-retrieval (Chain-of-Retrieval Augmented Generation). Beating it means higher deadline-constrained accuracy at equal channel uses and energy.
The strongest existing accuracy method is CoRAG with greedy decoding over text sub-queries, run over the same simulated link. CoRAG-8B is reported to beat IterDRAG and Search-o1-32B by over 10 exact-match points on MuSiQue and 2WikiMultihopQA, so it represents the accuracy ceiling for text-query iteration. IRCoT and DRAGIN are secondary text-query baselines. DRAGIN's reported F1 on 2WikiMultihopQA with LLaMA2-13B is 0.3931. All three get the same retriever, corpus, generator size where applicable, and the same radio simulator. Their per-hop transmission parameters are tuned by the same finite-blocklength allocator we use, so they are not handicapped by naive radio settings. Under this fair tuning, the remaining difference isolates the query representation.
The strongest latency-oriented method is L-RAG with an uncompressed float projection. It reports 0.352 accuracy on MuSiQue and about 2.8 s per query on 2WikiMultihopQA, against more than 20 s for IterRetGen (Optimizing Multi-Hop Document Retrieval Through Intermediate Representations). We found no reported result on Lacuna for any multi-hop QA method evaluated over a finite-blocklength wireless link, so every link-constrained number in this study is new.
Two ablations isolate the key ingredients. The first keeps our quantized query but allocates radio resources with Shannon capacity rather than the finite-blocklength formula. The second keeps finite-blocklength allocation but sends text queries compressed with a standard lossless text compressor at the same byte budget. Beating them means a statistically significant gain in "correct and on time" rate at matched energy, judged by paired bootstrap over questions.
Why not just…?
Why not just offload the whole pipeline to the server and send the question once? This is the most serious alternative. If the device can reveal its full local context and the server has spare generation capacity, it wins: one round trip, and no multi-hop traffic over the air. Our setting assumes the reasoning context is local and private, such as sensor state or personal memory, and that the server is a shared index without per-user generation. We will report full offload as a reference line, with its uplink cost when local context must be shipped. The comparison will show where the crossover lies rather than assume it away.
Why not just retrieve once with a much larger top-$k$? One round trip with many passages moves the cost into a large downlink packet. PAGE-RAG reports that widening the candidate pool floods the context with topical distractors, and that bridge evidence absent from the first pool cannot be recovered. CoRAG's MuSiQue recall gap (29.0% versus 47.1%) suggests that wide single-shot retrieval does not substitute for iteration. Our one-shot baseline sweeps $k$ at matched downlink channel uses, which settles this directly.
Why not just use Shannon-rate allocation or plain retransmissions, since the radio is fast anyway? Energy-Efficient Packet Scheduling With Finite Blocklength Codes shows that Shannon models underestimate loss and energy for short packets. Low-Latency Short-Packet Transmissions: Fixed Length or HARQ? shows that retransmission is not uniformly better under latency limits. Still, if the first-stage measurement finds the radio negligible, this alternative is correct. The Shannon-allocation ablation measures the gap rather than presumes it.
Proposed Experiments and Decisive Checks
All experiments use MuSiQue-Ans, 2WikiMultihopQA, and HotpotQA open-corpus. Evaluation uses exact match and F1, hop-level supporting-fact recall, and the primary metric: the fraction of questions answered correctly within deadline $D$ at energy budget $E$. We also report the 95th and 99th percentile end-to-end latency, following the tail focus of Ultra-Reliable Low-Latency Vehicular Networks: Taming the Age of Information Tail.
The radio is simulated with AWGN and Rayleigh block fading under pilot-assisted estimation. Pilot overhead is charged against the blocklength, as analyzed in Finite-Blocklength Bounds on the Maximum Coding Rate of Rician Fading Channels. Packet errors are sampled from normal-approximation and saddlepoint error probabilities, and the two are compared.
Three results decide the claim. The first is whether a crossover SNR exists below which quantized vector queries beat CoRAG-style text queries on deadline-constrained accuracy, and whether that crossover moves as finite-blocklength theory predicts when grant size changes. The second is whether finite-blocklength allocation beats Shannon allocation at matched energy, and whether its predicted packet-failure rates match simulated rates within a factor of two. The third is whether the exhaustively found optimum keeps radio error, retrieval miss, and deadline violation within one order of magnitude of each other, and whether the equalization heuristic comes within one accuracy point of exhaustive search. If the first check fails at every SNR, the representation idea is wrong. If the third fails, the cross-layer balance rule does not transfer to reasoning, and that is itself a reportable finding.
References
- Joint Sum Rate And Error Probability Optimization: Finite Blocklength Analysis
- Finite-Blocklength Information Theory
- Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
- Cross-Layer Transmission Design for Tactile Internet
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- Chain-of-Retrieval Augmented Generation
- DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models
- PAGE-RAG: Provenance-Aware Graph Evidence Promotion for Fixed-Budget Multi-hop Retrieval-Augmented Generation
- Energy-Efficient Packet Scheduling with Finite Blocklength Codes: Convexity Analysis and Efficient Algorithms
- Saddlepoint Approximations for Short-Packet Wireless Communications
- Alleviating Distortion Accumulation in Multi-Hop Semantic Communication
- Low-Latency Short-Packet Transmissions: Fixed Length or HARQ?
- Ultra-Reliable Low-Latency Vehicular Networks: Taming the Age of Information Tail
- Finite-blocklength bounds on the maximum coding rate of rician fading channels with applications to pilot-assisted transmission