Financial Risk Prediction with Multi-Round Q&A Attention Network

Zhen Ye, Yu Qin, Wei XuIJCAI 2020 · DOI

TL;DR

This paper introduces a novel "Multi-Round Q&A Attention Network" (MR-QA) to predict financial risk by analyzing earnings conference call transcripts. Unlike previous methods that focus on individual sentences or documents, MR-QA specifically models the crucial question-and-answer (Q&A) dialogue structure, demonstrating significant improvements in prediction accuracy.

What Problem Does This Paper Address?

Predicting financial risk, particularly stock volatility, is crucial for investors and company managers. Earnings conference calls are a key source of information for this prediction, as they reveal company performance and allow analysts to probe deeper. While prior research has used NLP techniques to extract features from these transcripts, they often overlook the inherent dialogue structure of the Q&A section. This dialogue, where senior managers respond to analysts' concerns, is a rich source of information that can reveal underlying risks. This paper aims to bridge this gap by developing a model that explicitly leverages the multi-round Q&A structure to improve financial risk prediction.

Methods

The MR-QA model is designed with a two-part architecture, mirroring the structure of earnings call transcripts: a presentation section and a Q&A section.

  • Presentation Section: This part uses a standard bidirectional LSTM (BiLSTM) to encode the textual information from the presentation.
  • Q&A Section: This is the core innovation. It involves several layers:
  • Sentence Encoding: BiLSTM is used to encode each sentence into a vector, creating sentence-level features.
  • Reinforced Sentence Selector (RSS): This novel component uses a hard attention mechanism and reinforcement learning to selectively filter out noisy or uninformative sentences within each Q&A round. This is crucial because not all sentences in a dialogue are equally relevant for risk prediction.
  • Reinforced Bidirectional Attention Network (RBAN): This component models the interaction between questions and answers within each round. It uses attention mechanisms to capture how questions influence answers and vice-versa, generating round-level features.
  • Round-level to Section-level Encoding: Another BiLSTM is used to aggregate the round-level features into section-level features for the entire Q&A.

Finally, the features from the presentation and Q&A sections are concatenated and fed into a regression layer to predict stock volatility. The model's parameters are optimized using a combination of policy gradient for the reinforcement learning components and back-propagation for the rest.

Datasets

The study utilizes a dataset of earnings call transcripts scraped from Seeking Alpha. The dataset comprises 6494 samples from 2015 to 2018, focusing on Standard & Poor 500 sample stock companies. The data is split chronologically into training (2015-2016), validation (2017), and testing (2018) sets to simulate real-world prediction scenarios.

Key Insights and Ideas

  • Dialogue Matters: The Q&A structure of earnings calls is a critical, previously underutilized, source of information for financial risk prediction.
  • Selective Attention is Key: A hard attention mechanism (Reinforced Sentence Selector) that actively filters sentences based on their informativeness significantly improves performance, suggesting that not all dialogue is equally valuable.
  • Interactions Reveal Risk: Modeling the bidirectional interaction between questions and answers (Reinforced Bidirectional Attention Network) captures nuanced insights that contribute to more accurate risk assessment.
  • Hard Attention Outperforms Soft: The paper found that a hard attention mechanism for sentence selection performed better than a soft attention approach, possibly because it forces a more definitive selection of informative content.

Key Results

The MR-QA model significantly outperforms several baselines, including past volatility, IR methods, and a strong prior model (ProFET), across different prediction spans (3, 7, and 15 days). For instance, it achieved a 47% improvement in Mean Absolute Error (MAE) for the 3-day span compared to past volatility. The paper also reports substantial improvements in Mean Squared Error (MSE) compared to ProFET, with reductions of 11% (3-day), 21% (7-day), and 32% (15-day) spans.

Limitations and Future Work

  • Data Imbalance: The paper notes that volatility is unbalanced across quarters, which they addressed by splitting data by year. However, this might still leave room for further exploration of temporal dynamics.
  • Reinforcement Learning Overfitting: The authors mention that overfitting can occur when training reinforcement learning components too late. They propose an earlier, staged training approach to mitigate this.
  • Future Work: The authors suggest that their findings could provide practical enlightenment for financial practitioners by offering a better way to understand financial disclosures. Further research could explore incorporating other modalities (like audio) or extending the dialogue analysis to other types of financial communication.

Summary updated

BibTeX
@inproceedings{ye2020financial,
  author = {Zhen Ye and Yu Qin and Wei Xu},
  title = {{Financial Risk Prediction with Multi-Round Q\&A Attention Network}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/631},
}
Figure