SURF: A System to Unveil Explainable Risk Relations between Firms

Yu-Hsiang Wang, Wei-Ning Chiu, Yi-Tai Hsiao, Yu-Shiang Huang, Yi-Shyuan Chiang, Shuo-En Wu, Chuan-Ju WangNAACL 2025

TL;DR

This paper introduces SURF, a novel system that uses deep learning to identify and explain complex risk relationships between companies by analyzing their financial reports. SURF goes beyond simply detecting connections, providing interpretable rationales for these relationships, making financial risk analysis more accessible and transparent.

What Problem Does This Paper Address?

Understanding the intricate risk relationships between companies is crucial for investors and financial professionals. Traditionally, this involves manually sifting through extensive financial reports, a process that is time-consuming, prone to subjective bias, and difficult to scale. Existing automated methods often lack interpretability, failing to explain why certain risks are linked, which hinders user trust in the "black box" of AI models. This paper addresses the need for a system that can automatically uncover and, importantly, explain these inter-firm risk connections in a transparent and user-friendly manner.

Methods

SURF employs a multi-faceted approach to identify and explain risk relations:

  • Domain-Specific Encoder: A BERT model is fine-tuned using a self-supervised contrastive learning approach. This training focuses on two views:
  • Chronological Similarity: Identifies paragraphs discussing similar events within the same firm based on date tokens.
  • Lexical Similarity: Detects paragraphs with overlapping text segments and similar phrasing, common in financial disclosures.
  • Mutual Risk Paragraphs (MRPs): Paragraphs identified as discussing similar risks between two firms are termed MRPs.
  • Risk Relation Score (RRS): A quantitative score is calculated as the ratio of MRPs to the total number of paragraphs from both firms, indicating the strength of the risk relation. A threshold-based approach is used to ensure retrieved paragraphs meet a minimum similarity level.
  • Rationale Explainer with LLMs: Large Language Models (LLMs), specifically ChatGPT (GPT-4o-mini), are used to summarize the identified MRPs. These summaries highlight shared risks and provide a coherent explanation for the connection between firms. The system also presents the original financial text alongside the LLM summaries to mitigate hallucination and ensure factual consistency.
  • Interactive Interface: SURF features a user-friendly interface that allows users to customize the analysis by selecting target years, companies, and the number of related firms to display. A graph visualization module displays firm relationships, with edge thickness indicating relation strength and node size representing market capitalization.

Datasets

  • Data Sources: The system is trained and evaluated using stock price data and Form 10-K filings from companies listed in the S&P 500 Index between 2018 and 2023.
  • Dataset Size: The dataset comprises 2,136 filings from 356 companies over a six-year span.
  • Benchmark Data: The paper compares SURF against baselines that include direct mentions of companies in each other's filings, GICS sector/industry classifications, and other dense encoders like DPR, Contriever, and FinBERT.

Key Insights and Ideas

  • Explainability is Key: The paper strongly emphasizes that for financial applications, understanding why a risk relation exists is as important as identifying the relation itself. SURF's LLM-powered rationale explainer directly addresses this.
  • Self-Supervised Learning for Financial Text: The use of contrastive learning on chronological and lexical similarities within financial documents is a novel way to train an encoder without requiring extensive manual annotation of financial risks.
  • Beyond Simple Mentions: Simply counting how many times one company is mentioned in another's filing is a weak indicator of risk relations. SURF's approach delves into the semantic content of the disclosures.
  • LLMs as Explanatory Tools: Leveraging LLMs to summarize and explain complex financial text is a powerful way to democratize access to insights that were previously only accessible to domain experts.
  • Dynamic Prompt Refinement: The system incorporates a feedback mechanism to continuously improve the LLM's summarization prompts, leading to more accurate and relevant explanations over time.

Key Results

  • Outperforms Baselines: SURF's two-view encoder consistently outperforms all baselines in identifying risk relations, as measured by the correlation between risk relation scores (RRS) and the correlation of daily stock returns (CAVDSR).
  • Quantitative Performance: The paper presents a table (Table 2) showing SURF's RRS-CAVDSR correlations, which are generally higher than those of Direct Mentions, GICS Sector/Industry, DPR, Contriever, and FinBERT across multiple years. For instance, in 2023, SURF achieved a correlation of 0.4191, while FinBERT achieved 0.3079 and Direct Mentions 0.0312.
  • Qualitative Validation: A case study on Nvidia and Wabtec demonstrated SURF's ability to uncover a strong, yet unexpected, risk relation related to supply chain disruptions, which was then validated by news reports and equity research.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that LLM-generated content is not a substitute for professional advice and users are advised to use it for reference purposes only.
  • While FinBERT, a finance-specific encoder, was included, it performed the worst among retrieval-based approaches, suggesting that general-purpose encoders fine-tuned for specific tasks might be more effective than domain-specific ones not explicitly trained for retrieval.
  • Future Work:
    • Integrate additional data sources such as news articles and analyst reports to provide more timely and diverse insights.
  • Explore the application of SURF's chronological similarity view to discover connections/relations between entities across different types of corpora (e.g., geopolitical risks).
  • Adapt the rationale explainer to other systems to enhance transparency in complex or opaque content.
  • Expand the system's application to different fields, fostering broader interdisciplinary insights.

Summary updated

BibTeX
@inproceedings{wang2025surf,
  author = {Yu-Hsiang Wang and Wei-Ning Chiu and Yi-Tai Hsiao and Yu-Shiang Huang and Yi-Shyuan Chiang and Shuo-En Wu and Chuan-Ju Wang},
  title = {{SURF: A System to Unveil Explainable Risk Relations between Firms}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure