Abstract
Causality represents the foremost relation between events in financial documents such as financial news articles and reports. Each financial causality contains a cause span and an effect span. Previous works proposed sequence labeling approaches to solve this task. However, sequence labeling models find it difficult to extract multiple causalities and overlapping causalities from text segments. In this paper, we explore a generative approach for causality extraction using an encoder-decoder framework and pointer networks. We use the FinCausal dataset for our experiments, where our proposed framework, CEPN, achieves competitive performance and establishes new state-of-the-art results.
Introduction
In financial analysis, identifying the relationship between events is crucial for understanding market dynamics and risk. Financial causality extraction involves identifying the "Cause" and "Effect" spans within a text segment. For instance, in the sentence "The stock dropped because of a sudden interest rate hike," a model must correctly identify the rate hike as the cause and the stock drop as the effect.
Traditionally, this task has been framed as a sequence labeling problem using BIO (Beginning, Inside, Outside) tagging. While effective for simple entities, sequence labeling struggles with the complexities of financial text, specifically when multiple causal relationships exist within a single sentence or when spans overlap. Furthermore, financial causality spans are often long—averaging 17 to 18 tokens—making exact match identification a significant challenge for traditional discriminative models.
The CEPN Framework
To address these limitations, we introduce CEPN (Causality Extraction with Pointer Networks). This framework shifts the paradigm from token-level classification to a generative sequence task.
Architecture
The model utilizes an encoder-decoder architecture designed to map input text to a sequence of causality coordinates:
- Encoder: A pre-trained BERT model (Base or Large) generates contextual embeddings for the input text. To enrich the representation, these embeddings are concatenated with part-of-speech (POS) tag embeddings.
- Decoder: An LSTM-based decoder generates a sequence of causality tuples. Each causality is represented as a tuple of four indices: $(c_{start}, c_{end}, e_{start}, e_{end})$, representing the boundaries of the Cause and Effect spans.
- Pointer Networks: Instead of selecting words from a fixed vocabulary, the decoder uses pointer networks to "point" to specific token indices in the original text. This allows the model to extract spans of any length and handle overlapping segments naturally.
Extraction Strategies
The authors evaluated two generation orders to determine if the sequence of extraction impacted performance:
- Cause-First (CF): The model identifies the Cause span before the Effect span.
- Effect-First (EF): The model identifies the Effect span before the Cause span.
Experimental Results
The CEPN framework was evaluated on the FinCausal 2020 and FinCausal 2021 datasets, which consist of complex text segments from financial news and reports. On average, these segments are 40 tokens long, with cause/effect spans often spanning across different sentences within the same segment.
Performance Metrics
The model's performance was measured using Token-level F1 and Exact-Match (EM) F1 scores. The results demonstrate that CEPN significantly outperforms previous state-of-the-art (SOTA) models like GBe.
- FinCausal 2020: The
CEPN_Large_EFmodel achieved a Token-level F1 of 0.874 and an EM F1 of 0.743, marking a 3.3% improvement in EM F1 over previous benchmarks. - FinCausal 2021: The
CEPN_Large_CFmodel reached a Token-level F1 of 0.892 and an EM F1 of 0.775.
The improvements were found to be statistically significant ($p < 0.001$), proving that the generative approach is fundamentally more capable for this specific task than traditional sequence labeling.
Discussion
The success of CEPN highlights several key insights into financial NLP:
- Generative vs. Discriminative: Causality is a structure rather than just a set of labels. By treating it as a sequence of tuples, the model can identify multiple relationships in one pass that a labeling model would find ambiguous.
- Handling Complexity: Because the model generates indices rather than assigning tags to tokens, it inherently solves the "overlapping span" problem. A token at index $i$ can be the end of a Cause in one tuple and the start of an Effect in the next.
- Order Independence: Interestingly, the model performed almost equally well regardless of whether it identified the Cause or the Effect first (CF vs. EF), suggesting the pointer mechanism is robust enough to capture the relationship from either direction.
Conclusion and Future Work
CEPN demonstrates that a generative approach using pointer networks is highly effective for financial causality extraction. While the model sets new performance standards, future work could address the computational latency inherent in decoder-based models and explore performance on broader, out-of-distribution financial datasets.
The authors have released their code and data at github.com/nayakt/CEPN.
Summary updated