Forecasting Beyond Historical Patterns with Event Driven Reasoning

This direction is merged into another direction: historical data dependency (exogenous shock vulnerability). Its last version stays readable and citable here.

Version 1 of 134 papers39 conceptsfrozen 2026-10-04

Traditional time series forecasting operates on a fundamental, often unspoken assumption: the future is a mirror of the past. Whether using classical statistical models like ARIMA or deep learning architectures like LSTMs, these systems search for recurring cycles and trends in historical numerical data.

However, as researchers like Xi Zhang and Xiaoyu Qin point out, this "historical mirror" shatters during "Black Swan" events—unprecedented shocks like global pandemics, sudden policy shifts, or geopolitical crises. When the underlying dynamics of a system change fundamentally, historical data becomes a map of a territory that no longer exists.

The Blind Spot of Purely Numerical Models

The core limitation of quantitative-only models is that they are inherently reactive. In the context of credit risk, Jiaqi Ma notes that models relying on historical datasets fail to provide proactive early warnings because they lack "causal intuition." They can tell you that a stock price is falling, but they cannot understand why it is falling if the cause is an external event—like a CEO scandal or a new regulation—that has never appeared in the training set.

This failure is evident across diverse domains:

  • Climate and Ecology: Yee Chun Tsoi observes that as climate change shifts weather patterns outside of historical norms, "analogue" forecasting (finding "historical twins" in past data) fails to signal record-breaking phenomena.
  • Supply Chains: Serina Chang argues that models ignoring the underlying logic of inventory and production cannot predict behavior when transaction schedules are disrupted by external shocks.
  • Public Health: During the COVID-19 pandemic, Yuan Yuan demonstrated that demand forecasting for essential goods only became accurate again once disease infection statistics were integrated into the models.

Bridging the Gap with Event-Driven Reasoning

To move beyond simple pattern matching, a new direction in research focuses on Event-Driven Reasoning. The goal is to give models "eyes" to see the world outside the spreadsheet.

The Value of Text (VoT) framework, introduced by Siyuan Wang and colleagues, represents a major shift in this direction. Instead of using Large Language Models (LLMs) merely as feature extractors, VoT treats them as active reasoners. By aligning numerical time series with external news descriptions, the model can "understand" that a sudden dip in a graph isn't just noise—it's a logical consequence of a specific real-world event.

Similarly, Xinlei Wang proposes a multi-agent system where LLMs "reflect" on past forecasting errors in the context of news. This allows the system to adjust its expectations for the future based on qualitative shifts in human behavior, such as social unrest or policy changes, which are invisible to traditional LSTMs.

From Prediction to Scenario Planning

If the future is truly unprecedented, perhaps the goal shouldn't be a single "correct" prediction, but a set of plausible futures. Shirin Sohrabi and the team behind IBM’s Scenario Planning Advisor (SPA) frame this as an AI plan recognition problem.

Instead of asking "What will happen?", the system asks "What drivers are currently active, and what paths could they lead to?" By connecting real-world "drivers" (like news or social media trends) to potential business impacts through causal models, SPA helps decision-makers prepare for "what-if" scenarios that have no historical precedent.

Multimodal Fusion and Frequency Stability

While text provides context, the technical challenge remains: how do we fuse erratic, qualitative news with precise, quantitative numbers?

  1. Modality-Adaptive Fusion: Ross Koval suggests treating both financial tables and long-form text (like earnings calls) as parallel time series. This allows the model to capture "earnings surprises" by spotting discrepancies between what the numbers say and what the executives are saying.
  2. Frequency-Domain Filtering: To handle the "distribution shift" that occurs during crises, Xihao Piao introduced the Time-Invariant Frequency Operator (TIFO). TIFO works in the frequency domain to identify stationary components of a signal that remain reliable even when the "time-domain" version of the data looks chaotic.

The Path Toward Human-Level Forecasting

The ultimate benchmark for these systems is "judgmental forecasting"—the ability to synthesize diverse, often conflicting information to predict complex outcomes like elections or economic shifts. Danny Halawi has shown that by combining specialized retrieval pipelines with LLMs, automated systems are now approaching the accuracy of professional human "superforecasters."

The transition from historical pattern matching to contextual reasoning marks a fundamental evolution in AI. By integrating the "why" of textual events with the "what" of numerical data, researchers are building systems that don't just look backward to see the future, but actually understand the world they are trying to predict.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_forecasting_beyond_historical_patterns_w_5a9e7fe5,
  title = {Forecasting Beyond Historical Patterns with Event Driven Reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/forecasting-beyond-historical-patterns-with-event-driven-reasoning/rdr_5a9e7fe5d5b24d79867ce190eb67fd9b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_forecasting_beyond_historical_patterns_w_5a9e7fe5_v1,
  title = {Forecasting Beyond Historical Patterns with Event Driven Reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/forecasting-beyond-historical-patterns-with-event-driven-reasoning/rdr_5a9e7fe5d5b24d79867ce190eb67fd9b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}