Introduction
Large Language Models (LLMs) have revolutionized information retrieval and reasoning, yet they remain plagued by a phenomenon known as the "lost in the middle" effect. When processing long contexts, models typically exhibit a strong position bias, demonstrating high accuracy for information located at the beginning or end of a prompt while struggling with data buried in the center. Traditionally, this bias has been viewed as a failure mode to be mitigated through random shuffling or architectural changes.
In this paper, we present Gold Panning, a novel black-box framework that reframes positional bias as a valuable diagnostic signal. By modeling context positions as noisy detectors and employing Bayesian active search, Gold Panning identifies relevant information—the "needle in the haystack"—with significantly higher efficiency than existing ensembling techniques.
The Gold Panning Framework
Gold Panning operates as an iterative Bayesian search process. Unlike standard methods that treat all context positions as equally informative, Gold Panning explicitly leverages the model's known strengths and weaknesses across different positions.
The framework consists of four primary stages:
- Calibration: The system first establishes a "positional diagnosticity profile" for the specific LLM. By placing known relevant documents at various positions, it measures the True Positive Rate (TPR) and False Positive Rate (FPR) for each location, identifying which parts of the context are most reliable.
- Bayesian Beliefs: A probability distribution (belief) is maintained for every document in the collection, representing the likelihood that it contains the required information.
- Signal Anchoring (Strategic Shuffling): In each iteration, documents are reordered based on current beliefs. Documents with the highest belief scores are placed in the most "diagnostic" positions—those where the model is most accurate. This "anchoring" strategy ensures that promising candidates receive the highest quality attention.
- Iterative Updates: The model is queried, and its citations are used to update the Bayesian beliefs. This cycle repeats until a confidence threshold is reached.
Theoretically, this greedy approach allows the system to identify a target document among $N$ candidates in $O(\log N)$ rounds, providing a robust mathematical foundation for its efficiency.
Experimental Evaluation
The researchers evaluated Gold Panning using the FLenQA (Factuality in Long-context QA) benchmarks, specifically the MonoRel (monotonic transitive reasoning) and PIR (compositional reasoning) tasks. These were structured as Multi-Document Question Answering (MDQA) problems with up to 500 documents. Testing was conducted across the Gemma-3 (12B, 27B) and OLMo 3 (7B, 32B) model families.
Results and Discussion
The core insight of Gold Panning is that model biases are predictable regularities rather than random errors. By treating "sharp" zones of the context window as high-quality sensors, the framework achieves remarkable efficiency gains.
Key findings include:
- Efficiency: Gold Panning matches the accuracy of Permutation Self-Consistency (PSC) while requiring 30–65% fewer LLM queries. On Gemma-3-12B, near-perfect F1 scores were achieved in just 4 iterations, whereas the baseline required 8.
- Anchoring vs. Exploration: The "GP-Belief" (anchoring) strategy consistently outperformed uncertainty-reduction strategies (GP-Entropy). Probing ambiguous documents often introduced noise, while anchoring likely signals accelerated the separation of relevant information from distractors.
- Robustness: The method proved highly resilient to calibration errors. As long as the relative ranking of positions (best vs. worst) remained accurate, the performance gains persisted even if the exact TPR/FPR estimates were slightly off.
Conclusion and Future Directions
Gold Panning demonstrates that LLM limitations, such as positional bias, can be systematically exploited to improve system performance. By transforming a "bug" into a "feature," the framework provides a powerful tool for long-context reasoning.
While highly effective, the iterative nature of the process introduces latency compared to single-query methods. Future work may explore using internal model states to refine belief updates, potentially reducing the need for multiple full-text queries while maintaining the benefits of the Bayesian framework. Additionally, extending the "signal anchoring" dynamics to scenarios with multiple relevant documents remains a promising area for investigation.
Summary updated