Geographic Drift as an Early Warning for Retrieval Cascades over Maps and Reports

When a search agent answers a multi-step question, everything downstream depends on what it retrieves first. If the first retrieved fact is wrong, later reasoning cannot recover. In text-only settings this is now measured precisely. Agents that retrieved the gold evidence were right 73.8% of the time, those that missed it were right 2.2% of the time, and failed runs kept searching two to three times longer without finding anything new (Diagnosing Search Behavior and Failure Modes in Long-Horizon Search Agents). This proposal studies questions about places, where evidence is scattered across planning reports, environmental assessments, and the maps inside them. For these questions there is a cheap, label-free signal of whether an agent has lost the thread. That signal is how far, in physical kilometres, the retrieved evidence sits from the place the question is about.

The signal draws on a well-established empirical regularity called distance decay: the probability that two things are linked falls off steeply, usually as a power law, with physical distance. Social ties, conversations, and citations all show it (Multiple Location Profiling for Users and Relationships from Social Network and Content; Multi-dimensional Conversation Analysis across Online Social Networks). Our claim is that the evidence chains behind geographic questions decay the same way. If so, an agent whose retrieved evidence drifts outward, far beyond the decay curve fitted from correct chains, is probably in a failing trajectory. The same fitted curve can be used to rerank candidates and to stop searching early. What is new is a calibrated, soft geographic prior, fitted from the corpus's own correct evidence chains, used both to detect and to prevent cascading retrieval failures. A second new element is that the location of each map is itself extracted from the map's pixels, so errors in reading maps enter the prior and can be measured.

Core Claim

We make three predictions. First, in multi-hop geographic questions over a pooled corpus of reports and maps, failed trajectories show measurably larger geographic drift from the question's anchor location than successful ones, and they show it early, before the "wasted tail" of redundant search begins. Second, adding a log distance-decay term to the retrieval score reduces the share of failures in which the needed evidence never entered the agent's context, and it does better than a hard geographic filter specifically on questions whose evidence legitimately spans borders or scales. Third, the benefit depends on how accurately the system extracts locations from map images, so the multimodal extraction step is a quantified link in the cascade rather than a preprocessing detail.

A tension in the file has to be faced up front. FRIEDA benchmarks multi-step map reasoning and finds near-identical accuracy whether models receive the correct map or must retrieve it from its source document. The authors conclude that spatial reasoning, not retrieval, is the bottleneck. We take this seriously. Their retrieval is within a single report, where few candidate maps exist. Our hypothesis is that retrieval becomes the bottleneck once the corpus is pooled across 210 reports and 32 countries and questions chain across documents. If that hypothesis is false, the proposal's premise is false, and the first experiment is designed to find out.

Research Program

The prior. Each evidence unit $u$ in the corpus is a paragraph, table, or map. Each gets a location $\ell(u)$. For text, this is the geocoded place mentions in the unit. For maps, it is the geographic extent read off the image: place labels, the scale bar, and legend-declared regions. Each question gets an anchor location $a$, initially taken from the first retrieved hop. Retrieval scores become

$$s(q,u) = s_{\text{sem}}(q,u) + \lambda \log K\big(\mathrm{dist}(a,\ell(u))\big), \qquad K(r) = (1 + r/r_0)^{-\gamma}.$$

Here $s_{\text{sem}}$ is the base retriever's semantic score, $\mathrm{dist}$ is great-circle distance in km, and $r_0$ and $\gamma$ are a distance scale and a decay exponent. These are fitted by maximum likelihood on gold hop pairs from training questions, separately for each coarse question type. $\lambda$ is a weight tuned on development data. We take the power-law-with-core form from the social-tie literature (Multiple Location Profiling), because a power law allows the occasional legitimate long-distance link that an exponential or hard radius would cut. The fit is allowed to return $\gamma \approx 0$ for question types that are genuinely non-local. This matters because globally spread users are exactly where median-of-neighbours geolocation breaks down (Geotagging one hundred million Twitter accounts with total variation minimization). That paper places users at the $l_1$ median of their contacts' locations and reaches a 6.38 km median error. We borrow its robust-median idea to define the anchor and use the decay function for scoring, not placement.

Anchor uncertainty. The usual root of a cascade is a wrong first foothold, as HopRetriever shows for Wikipedia hyperlink paths. A wrong anchor would make the prior push the agent further in the wrong direction. To guard against this, the system keeps two or three anchor hypotheses, each weighted by the support it has from retrieved evidence. It re-anchors when the robust median of recent evidence locations moves away from the current anchor.

The drift detector and stopping rule. Drift is the average negative log-likelihood under $K$ of the last few retrieved units' locations. When drift exceeds a threshold calibrated on training trajectories, the agent takes one of two actions: it re-anchors, or it stops and abstains. This gives a spatial alternative to the text-only learned stopping in FrugalRAG, which trains a 7B model with reinforcement learning to stop near the optimal number of searches but has no notion of where its evidence lies.

Map-location extraction. Map extents come from a vision-language model prompted to read labels, the legend, and the scale bar. Its known weaknesses are legend misreading, which accounts for 25.6% of FRIEDA errors, and poor metric distance reading, at 20–35% accuracy in FRIEDA. These are measured as a source of error in the prior, not assumed away. Extraction of text regions follows the layout-aware detection of Detect, Retrieve, Comprehend. That work segments PDF pages into regions with a document image transformer, reaching 75.34 paragraph-extraction F1 against 62.73 for Tesseract, but it ignores figures. We extend that pipeline to map regions.

The corpus. We pool all 17,030 FRIEDA maps and the text of their 210 source reports into one retrieval corpus. We then write cross-document questions in the style of MultiModalQA, which chains text, tables, and images through discrete hops, but here the hops run between paragraphs and maps of different reports. Some questions are anchored locally. A deliberately sizeable fraction require cross-border or multi-scale evidence. Without that fraction, a country filter would win trivially. For text-only transfer we also use the subset of BrowseComp-Plus questions that have a geocodable anchor, since its human relevance judgments let us label each failure as either needed evidence never retrieved or evidence retrieved but misused.

Baselines

The naive reference is a standard ReAct-style agent: one that alternates reasoning steps with search calls, using the same base retriever and vision-language reader, with no geographic term and a fixed hop budget. This tells us how often pooled-corpus cascades happen at all. Single-document FRIEDA numbers serve as the context ceiling. Gemini-2.5-Pro reaches 38.20% and GPT-5-Think 37.20% accuracy against 84.87% for humans when given the correct maps (FRIEDA). Nothing on the pooled corpus can be expected to exceed those numbers by retrieval alone. There is no reported result on Lacuna for any system on a pooled cross-report FRIEDA corpus, so the naive agent establishes that number.

The strongest existing method for this question is FrugalRAG-style learned stopping combined with Retrieval-Grounded Voting. Retrieval-Grounded Voting, from Beyond Confidence, samples several agent runs and weights each one's answer by how much of it overlaps lexically with the documents that run retrieved. Together these two pieces form the best published remedy for cascades and wasted search. FrugalRAG reports 68.47% model-based-evaluation accuracy with 2.05 searches on HotpotQA against 64.20% for ReAct, and over 20% zero-shot on BrowseComp-Plus against 16.39% for DeepSeek-R1-0528. Retrieval-Grounded Voting with 4 runs matches confidence voting with 8. We retrain the stopping policy on our corpus with the same roughly 1,000-example budget the FrugalRAG paper uses, apply Retrieval-Grounded Voting with the same number of runs as our method, and match total search calls per question. Beating this baseline means a statistically significant drop in the share of failures where the needed evidence never entered context, plus higher end-to-end accuracy at equal or lower search cost. The gain must hold on the cross-border subset and not only on local questions.

Three further baselines target the proposal's own ingredients. The first is a hard filter that keeps only candidates in the anchor's country, or within a radius tuned on development data. The second is location-in-query, which appends the anchor place name to every search. The third is a set of ablations: $\gamma$ fixed at 0, so the prior is off; $\gamma$ fixed rather than fitted per question type; a single anchor rather than several; and oracle map locations in place of extracted ones. The oracle ablation isolates how much of the remaining error is caused by multimodal extraction. All variants share the same retriever, reader, tuning budget, and search budget.

Why not just…?

Why not just let the agent search longer? In text, search effort is decoupled from success. Failed runs are two to three times longer, and 66–85% of their context fills with search snippets (Diagnosing Search Behavior). FrugalRAG's exploratory policy, averaging 5.99 searches, is beaten by its pruned 2.05-search policy. We still run accuracy as a function of search budget on the pooled corpus, because maps might reward exploration differently from text.

Why not just filter by country or a fixed radius? This is the real threat. If most questions are local, a hard filter will match the soft prior at lower complexity. Our answer is empirical, not rhetorical. We report results separately on local and cross-border questions. We predict a tie on the local subset and a win only where a hard cutoff discards legitimate long-distance links. If the soft prior does not beat the filter on cross-border questions, we will recommend the filter and drop the decay prior.

Why not just give a frontier vision-language model the whole document? Even with the correct maps provided, frontier models score about 38% on FRIEDA. On MapIQ, a single-map benchmark, they average 40.75% against 93.23% for humans and are fragile to changes in map design. Long context does not fix extraction, and it cannot hold 210 reports at once. We include a long-context run on the single-document subset as a reference.

Why not just add the place name to the query? Dense retrievers do match place-name tokens. However, maps often label sub-regions that never appear in the query, and a name match carries no information about distance. The location-in-query baseline measures this directly, and it is also combined with our prior to check whether the two are complementary.

Proposed Experiments and Decisive Checks

The first check tests the premise. We replicate FRIEDA's finding that accuracy with the correct map is about the same as accuracy after within-document retrieval, then measure the drop on the pooled corpus. If the pooled corpus shows no meaningful gap, retrieval is not the map bottleneck, and the project narrows to text-geography questions on BrowseComp-Plus.

The second check is a zero-training diagnostic on existing agent trajectories. For geocodable BrowseComp-Plus questions and for pooled-corpus runs, does drift in the first three hops predict eventual failure, measured by area under the ROC curve, and does it do so better than trajectory length? If drift carries no signal before the wasted tail begins, the detector idea fails regardless of reranking gains.

The third check applies the fitted prior and the drift-triggered stopping rule against all baselines above. The metrics are end-to-end accuracy, the share of failures where needed evidence never entered context, and searches per question, all reported separately for local and cross-border questions.

The fourth check substitutes oracle map locations and then deliberately degrades legend and scale reading. We plot downstream accuracy against map-location error, which directly measures how map-reading mistakes propagate into retrieval failure. A flat curve would mean multimodal extraction is not on the critical path for this pipeline, and we would report that as a finding.

Finally, we report the fitted $\gamma$ values by question type. Values near zero for global questions, and steep values for municipal planning questions, would show that the prior is learning real structure in the corpus rather than an arbitrary distance penalty.