Thinking Fast, Trading Late: System One Decision Models vs the Market

Amir Dehkordi, Alireza Kargarzadeh, Nariman Khaledian, Navid Parvini, Sid Ghatak, Arman KhaledianarXiv 2026 · arXiv

This paper, titled "Thinking Fast, Trading Late," provides a sobering reality check for the burgeoning field of using Large Language Models (LLMs) to predict stock market movements. While many studies claim LLMs can "beat the market" based on historical news, the authors argue that most of these gains are mirages created by look-ahead bias, memorization, and the simple fact that markets move faster than traders can read.

TL;DR

The authors introduce OLIVIA, a rigorous protocol to prevent backtesting bias, and apply it to AI decision models reading S&P 500 news. They find that while LLMs can predict overnight price moves, these moves are already "priced in" by the time a trader can act at the market open, and no tested strategy survives real-world trading costs.

What Problem Does This Paper Address?

The core challenge is the "backtesting delusion." In empirical finance, it is notoriously easy to find a signal that would have worked in the past but fails in the future. LLMs introduce specific risks:

  1. Look-ahead Bias: Models may have been trained on data that includes the "future" relative to the test period.
  2. Memorization: Even if a company name is masked, a model might recognize a specific event (e.g., a massive merger) and "remember" that the stock went up.
  3. Selection Bias: Researchers often try thousands of variations and only publish the one that happened to work.

Furthermore, the authors investigate market efficiency: even if a model correctly identifies news as "good," has the market already reacted by the time the news is public?

Methods

The authors propose the OLIVIA protocol, a set of six safeguards designed to make financial LLM research reproducible and honest:

Table 1: The OLIVIA protocol: six safeguards for testing language-model signals on financial news, what each guards against, how this study implements it and what a replication needs. The initials of the safeguards give the name.
  • AI Decision Models: Instead of using generative LLMs (like ChatGPT) to write summaries, they use non-generative models that return specific probabilities for typed questions (e.g., "Will this stock rise by more than 1%?").
  • Laya-Fin: An open-weights model (based on ModernBERT) fine-tuned specifically on 2024–2025 financial news and returns.
  • Identity Masking: They replaced company names, tickers, and dates with generic placeholders to test if models were truly "reading" the news or just "remembering" the company.
  • Decision Points: They tested models at three times: D0 (before the market opens), D1 (30 minutes after the open), and D2 (after the close).
Figure 1: The design: every choice was made on 2025 validation data, the 2026 test was sealed until the plan was frozen, and every model was released after the sample ended. (a) The calendar; the time axis changes scale at the break, from months to days. The test outcomes were released only after the freeze. The forward test, pre-registered on 2026-10-29 and not yet run, uses news published from 2026-10-22 on. (b) One trading session. At D0 (09:15 ET) a model reads the news published since the previous close; at D1 (10:00) and D2 (after the close) it also sees the reaction so far. The overnight move zon runs from the previous close to the open, so it includes after-hours and pre-market trading; the move after the open zid runs from the open to the close; and y2 runs from the next open to the close of t + 5. The convergence ladder of Section 6.2 runs to Ct+20.

Datasets

  • Universe: Point-in-time membership of the S&P 500.
  • News Feed: NewsWitch, a commercial feed providing titles and snippets with precise timestamps.
  • Time Periods: Training (July 2024–June 2025), two validation quarters in 2025, and a blinded test period (January–July 2026).

Key Insights and Ideas

  • Masking is not Anonymity: In a "matched-distractor" test, commercial models correctly identified the masked company in 62% to 74% of cases (against a 28% baseline). The models don't need the name; they recognize the unique "fingerprint" of a company's news (e.g., a specific drug trial result).
  • The "Overnight" Trap: Commercial models showed a strong correlation with overnight returns (the move from yesterday's close to today's open). However, they showed almost zero ability to predict the intraday move (open to close). This suggests the market incorporates news into the price during pre-market trading, leaving nothing for a trader who waits for the official open.
  • System One vs. System Two: Borrowing from psychology, the authors suggest LLMs act like "System One"—they can quickly categorize news as good or bad. However, they lack "System Two" reasoning—they cannot judge if the market overreacted or underreacted to that news.
  • Validation Decay: A signal that looked highly significant in validation (IC of 0.065) dropped precipitously in the blinded test (IC of 0.020), highlighting how easily researchers can fool themselves without a "frozen" analysis plan.

Key Results

  • Information Coefficient (IC): Commercial models achieved an IC of 0.074 to 0.077 for overnight moves, but this fell to near zero for moves occurring after the trader could actually enter a position.
  • Economic Significance: No pre-registered trading strategy achieved a positive return after accounting for a modest 10 basis point (0.1%) trading cost.
  • Variance Explanation: News readings explain only 0.25% to 0.82% of a stock's daily variance. The vast majority of stock price movement remains "noise" to these models.

Limitations and Future Work

  • Large-Cap Bias: The study focused on the S&P 500. The authors acknowledge that news might diffuse more slowly in smaller, less-followed stocks, potentially leaving more room for LLM-based profits.
  • Training Data Transparency: Because commercial models (like GPT-4 or Claude) have undisclosed training data, the authors could not fully rule out that the models had "seen" the test period during their pre-training.
  • Future Forward Test: The authors have pre-registered a "forward test" to be run on news published after the models were released (late 2026/early 2027) to provide the cleanest possible test of memorization.

Summary updated

Abstract

Large language models (LLMs) read company news at scale, and backtests suggest that their readings predict stock returns. We ask how much of that predictability survives a test that rules out look-ahead, memorization and selection. We propose OLIVIA, a protocol for such tests: point-in-time inputs with company identities masked, selection on validation data only, an analysis plan frozen by cryptographic hash, and a test period unblinded once through a logged lock. We apply it to AI decision models, non-generative language models that return probabilities for typed questions, reading news about S&P 500 stocks: an open-weights model we fine-tuned on 2024-2025 data, three commercial models and standard baselines. The fine-tuned model's signal replicated on two validation quarters (daily information coefficients 0.055 and 0.065) but fell to 0.020 (t = 1.7) on the blinded January-July 2026 test. The commercial models' readings line up with the overnight move to the open (information coefficient 0.074 to 0.077) but not with the move after it, so the news is priced by the time a trader can act on it at the open. No model can tell over- from under-reaction, and no pre-registered strategy survives costs and a deflated Sharpe ratio. Commercial models identify masked companies in 62% to 74% of cases against a 28% baseline: masking hides names, not companies. The market reads the news first.
BibTeX
@misc{dehkordi2026thinking,
  author = {Amir Dehkordi and Alireza Kargarzadeh and Nariman Khaledian and Navid Parvini and Sid Ghatak and Arman Khaledian},
  title = {{Thinking Fast, Trading Late: System One Decision Models vs the Market}},
  year = {2026},
  eprint = {2611.03985},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03985},
}
Figure