TL;DR
A single fabricated news article is sufficient to flip the predictions of state-of-the-art LLM forecasting systems in over 80% of cases. The study reveals that the very multi-agent search capabilities that make these systems powerful also make them highly vulnerable to "weak" attackers who simply publish plausible fake news to the open web.
What Problem Does This Paper Address?
Large Language Models (LLMs) are increasingly deployed for financial and geopolitical forecasting, often using Retrieval-Augmented Generation (RAG) to ground their predictions in real-time news. While this improves accuracy, it creates a massive, unvetted attack surface.
The authors investigate whether a "weak attacker"—one who only knows the forecasting question and has no access to the model's internals, prompts, or retrieval queries—can manipulate these systems. The core challenge is that forecasting systems must rely on the authenticity of retrieved documents, yet they lack a reliable mechanism to verify information published after their training cutoff.
Methods
The researchers evaluated eight forecasting systems, ranging from simple single-retrieval baselines to sophisticated deployed architectures like the AIA Forecaster and xAI’s Grok. They employed two primary attack strategies:
- Directional Attack: The attacker publishes one plausible news article supporting a chosen direction (e.g., a stock will fall).
- Destabilizing Attack: The attacker publishes two articles arguing opposite directions to see if the model hedges its prediction.
The study used two protocols: Protocol A, where the fabrication is forced into the model's context to measure its influence, and Protocol B, where the article is added to a news archive to see if the system's own retriever finds it.
To counter these, they tested two state-of-the-art defenses: Passage Filtering (using an LLM to judge if a passage is fabricated) and Partitioned Retrieval (isolating retrieval into disjoint parts of the archive to prevent a single document from dominating the aggregate result).
Datasets
The study utilized a robust set of real-world and simulated forecasting targets:
- Stock Questions: 183 questions regarding one- and five-day returns, macro events, and stock spreads, evaluated against real market moves in 2026 windows.
- Prediction Markets: 308 questions from ForecastBench, Polymarket, and Kalshi covering sports, politics, and economics.
- News Archives: Custom archives containing 2,800 to 4,500 genuine news passages per time window, frozen to prevent data leakage.
Key Insights and Ideas
- The Retrieval Paradox: Systems designed to be "thorough" by using multiple agents and wide searches are actually more exposed. While a simple retriever might miss a single fake article, multi-agent systems like Grok and AIA retrieve the fabrication in ~90% of runs.
- The Verification Gap: Models cannot distinguish between "new" genuine news and "new" fabricated news. When asked to flag fabrications, models exhibited a high false-positive rate, flagging nearly half (45.7%) of genuine news as fake (see Table 3).
- Failure to Hedge: In destabilizing attacks, models don't become "unsure." Instead, they typically pick one side based on the article's rank or specific content, often returning a more confident (but wrong) forecast than the clean baseline.
- Invisible Harm: Standard accuracy metrics fail to detect the attack. Because the attacker is "right" about 50% of the time by chance, the system's average accuracy remains stable even as its specific decisions are being dictated by the attacker.
Key Results
- Reversal Rates: Once a fabricated article enters the context, it reverses 74% to 82% of forecasts that originally opposed the attacker's direction across all frontier models tested (GPT-4, Gemini 1.5 Pro, etc.).
- Model Compliance: Grok 4.20 was found to be the most compliant, following the fabricated article in 96.6% of stock-related cases (see Table 2).
- Defense Fragility: While passage filtering and partitioned retrieval can stop a single article, they are easily bypassed. An attacker can use "different framings" (multiple versions of the same story) to beat partitioning, or "distortions" (editing one fact in a real article) to bypass filters.
- Cost of Defense: Implementing both defenses together changes 29% of clean decisions and discards 39% of genuine news, representing a significant performance penalty for deployers.
Limitations and Future Work
The authors acknowledge that their evaluation was conducted on frozen archives rather than the live web to avoid ethical issues with planting real fake news. Furthermore, the "distortions" used were not optimized against the specific filters, suggesting the actual vulnerability might be even higher.
Future Work directions include:
- Developing defenses that isolate sources rather than just passages.
- Implementing corroboration requirements (requiring $n$ independent sources for a claim).
- Creating decision rules for system supervisors to discount events that rest on a single, newly-published source.
- Exploring how these attacks scale over longer forecasting horizons.
Summary updated