Introduction
Predicting corporate performance is a fundamental challenge in finance. While traditional models rely heavily on numerical data like stock prices and financial ratios, they often miss the nuanced signals embedded in long-form textual documents and broader macroeconomic trends. Furthermore, financial analysts' consensus estimates are frequently subject to behavioral biases and may fail to integrate complex, multi-modal information effectively.
This paper introduces a novel multimodal framework designed to predict earnings surprises by treating both financial tables and textual data as historical time series. By moving beyond isolated data silos and incorporating temporal context over multiple quarters, the proposed model offers a more holistic view of a company's trajectory.
Methodology
The core of the framework is a sophisticated architecture that processes diverse data types through specialized encoders before fusing them for a final prediction.
Specialized Encoders
- Time Series Mixer (TSM): For numerical inputs—including firm-level financial characteristics and macroeconomic indices—the authors employ an all-MLP architecture. TSM performs alternating feature mixing across time and channel dimensions, providing an efficient alternative to Transformers for multivariate time series.
- Multi-Stage Textual Feature Extraction (MTFE): To handle high-dimensional text like earnings call transcripts (ECT) and quarterly reports (QFR), a three-stage process is used. First, a BigBird model undergoes domain adaptation via masked language modeling and contrastive learning (LC-DiffCSE). Second, it is fine-tuned to predict earnings surprises from single documents. Finally, embeddings are extracted to form a "textual time series."
Cross-Modality Fusion (CMF)
Rather than simple early concatenation, the model utilizes a "late fusion" approach. Each modality is processed by its own adaptive encoder. A cross-modality multi-head attention (MHA) mechanism then allows these different channels to interact based on their temporal similarity, ensuring that the most relevant signals from each modality are prioritized.
Data and Experimental Setup
The study utilizes a comprehensive dataset covering the period from 2010 to 2020, focusing on large US public companies within the MSCI USA Index. The data includes:
- Textual Modalities: Earnings conference call transcripts (ECT) and Management Discussion and Analysis (MDA) sections from quarterly reports (QFR).
- Tabular Modalities: 15 firm-level financial characteristics and autoregressive (AR) lags of earnings surprises.
- Macroeconomic Data: 8 key indices, such as inflation and oil prices.
The target variable is the Standardized Unexpected Earnings (SUE), which measures the deviation of actual earnings from analyst consensus estimates.
Results and Performance
The experimental results demonstrate that integrating multiple modalities significantly improves forecasting accuracy. The multimodal model, combining AR, financial, textual, and macroeconomic data, achieved the lowest Mean Squared Error (MSE) of 1.42.
Key Findings
- The Power of Multimodality: The full multimodal model outperformed the best unimodal baseline (ECT at 1.60) and standard tabular-only approaches.
- Text as a Forward-Looking Signal: During the economic disruption of 2020, numerical data—which is inherently backward-looking—struggled to predict the rapid recovery of certain sectors. In contrast, textual models captured executive sentiment regarding emerging trends like remote work, leading to more resilient predictions.
- Unscripted vs. Scripted Text: Earnings call transcripts (ECT) proved more predictive than formal quarterly reports (QFR). This suggests that the spontaneous nature of analyst Q&A sessions provides more authentic insights than the highly regulated and "boilerplated" language of SEC filings.
- Temporal Context: Utilizing a history of 8 quarters provided a consistent performance boost over models looking only at the most recent quarter, highlighting the importance of tracking the evolution of a company's narrative.
Economic Impact
To validate the model's practical utility, the authors conducted a simulated trading exercise. The multimodal strategy achieved a Net Sharpe Ratio of 1.03, substantially higher than the tabular-only strategy (0.73) and the autoregressive baseline (0.47). This indicates that the model's predictive gains translate into meaningful economic advantages.
Conclusion and Future Directions
This research demonstrates the significant value of treating textual data as a time series and fusing it with traditional financial metrics. While the current model focuses on large US companies on a quarterly basis, future work could expand the geographic scope or increase the update frequency by incorporating mid-quarter news and macro data. Additionally, exploring the capabilities of newer Large Language Models (LLMs) with expanded context windows presents a promising avenue for further enhancing textual feature extraction in financial contexts.
Summary updated