Fact Check: Analyzing Financial Events from Multilingual News Sources

Linyi Yang, Tin Lok James Ng, Barry Smyth, Ruihai DongarXiv 2021 · arXiv

Abstract

Financial markets are increasingly global, yet news analysis remains siloed by language and geography. This paper introduces FactCheck, a multilingual platform that aggregates news from English and Chinese sources to verify financial events. By leveraging unsupervised clustering for event extraction and an adversarial transformer-based model (BERT-WWM+adv) for fact-checking, the system achieves an accuracy of 89.4% in predicting Mergers and Acquisitions (M&A) outcomes, significantly outperforming standard deep learning baselines.

Introduction

The financial sector generates a staggering volume of data daily. For investment analysts, maintaining a holistic view of market-moving events is hindered by two primary factors: the language barrier and the lack of scalable verification. News from a single region often provides a narrow perspective; for instance, a scandal involving a Chinese firm might be reported with different nuances in local media versus Western outlets. Furthermore, traditional fact-checking relies on human experts, a process that is too slow and expensive to match the 24/7 digital news cycle.

System Architecture

The proposed FactCheck system addresses these gaps through a two-pronged approach built on the AllenNLP library and the Flask framework.

The FactCheck system architecture, illustrating the pipeline from multilingual news crawling to final fact-checking.

Multilingual News Aggregator

The system crawls news from diverse sources, including Reuters, Fox, and Sina. Using the Baidu API, articles are translated into a bilingual format. To manage redundancy, an unsupervised news clustering model groups related stories across languages into a single "event" based on hashtags and content analysis.

Adversarial Fact-Checker

The core of the system is a Whole Word Masking BERT (WWM-BERT) architecture. To enhance robustness against misinformation and noise—common in financial reporting—the researchers implemented adversarial training using Projected Gradient Descent (PGD). This technique introduces small perturbations to input embeddings during training, forcing the model to learn generalized features rather than over-fitting on specific keywords.

Experimental Setup

The study utilized two primary datasets:

  1. General News Corpus: 183,991 articles (116,390 English; 67,601 Chinese) from early 2020.
  2. M&A Case Study Dataset: 4,098 instances of Merger and Acquisition deals sourced from the Zephyr database (2007–2019), used to evaluate the fact-checker's predictive capabilities.

Results and Evaluation

The adversarial transformer model (BERT-WWM+adv) was evaluated against several baselines, including CNN-Text, BiGRU, and standard BERT-WWM.

Performance comparison showing the proposed BERT-WWM+adv model outperforming all baselines in MCC, Accuracy, and F1 score.

As shown in the results, the proposed model achieved an accuracy and F1 score of 89.4%, a significant improvement over the standard BERT-WWM (87.4%) and RNN-based models (~84%). Notably, the Matthews Correlation Coefficient (MCC) reached 0.788, demonstrating the model's reliability in binary classification tasks for complex financial events.

Discussion

The integration of cross-lingual reports provides a "360-degree" view of events, where local reports may offer early warnings or granular details absent from international wires. The success of adversarial training (PGD) highlights that robustness is crucial in financial contexts where news can be intentionally misleading.

Future work will focus on expanding the system to include more languages and validating its transferability to other financial events, such as earnings surprises or regulatory changes. Additionally, the researchers plan to explore the integration of Large Language Models (LLMs) to further refine reasoning and mitigate potential hallucinations in the fact-checking process.

Summary updated

Abstract

The explosion in the sheer magnitude and complexity of financial news data in recent years makes it increasingly challenging for investment analysts to extract valuable insights and perform analysis. We propose FactCheck in finance, a web-based news aggregator with deep learning models, to provide analysts with a holistic view of important financial events from multilingual news sources and extract events using an unsupervised clustering method. A web interface is provided to examine the credibility of news articles using a transformer-based fact-checker. The performance of the fact checker is evaluated using a dataset related to merger and acquisition (M\&A) events and is shown to outperform several strong baselines.
BibTeX
@misc{yang2021fact,
  author = {Linyi Yang and Tin Lok James Ng and Barry Smyth and Ruihai Dong},
  title = {{Fact Check: Analyzing Financial Events from Multilingual News Sources}},
  year = {2021},
  eprint = {2106.15221},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.15221},
}
Figure