TL;DR
This paper introduces a novel two-stage framework for document-level event relation extraction (ERE) that significantly improves efficiency and performance. By first using a "retriever" to identify likely related event pairs and then a "cross-encoder" to classify their relationships, the approach tackles the quadratic complexity and skewed data distribution challenges inherent in existing methods.
What Problem Does This Paper Address?
Event Relation Extraction (ERE) is crucial for building comprehensive knowledge graphs by identifying temporal and causal links between events. However, existing methods often struggle with document-level ERE (DERE) due to two main issues:
- Computational Inefficiency: Pairwise comparison of all event pairs in a document leads to quadratic time complexity, making it slow for long texts.
- Skewed Data Distribution: Most event pairs in a document are unrelated, creating a heavily imbalanced dataset that makes it difficult for models to learn effectively.
This paper aims to address these challenges by proposing a more efficient and accurate approach for DERE.
Methods
The authors propose a novel pruning-based two-stage framework for DERE:
- Stage 1: Candidate Event Pair Retrieval: This stage employs a retriever model (specifically, a bi-encoder) to efficiently sift through event mentions. It embeds event mentions into a latent space and identifies pairs that are likely to have a relationship. This step effectively prunes the vast number of unrelated event pairs, reducing the computational burden for the next stage.
- Stage 2: Pairwise Classification: The retrieved candidate event pairs from Stage 1 are then fed into a cross-encoder model for fine-grained relation prediction. This stage uses either discriminative models (e.g., RoBERTa) or generative models (e.g., T5) to classify the specific temporal or causal relationship between the event pairs.
This two-stage approach aims to improve efficiency by reducing the number of pairs the expensive cross-encoder needs to process and to mitigate the skewed data distribution by focusing on potentially related pairs.
Datasets
The authors evaluate their framework on three well-established datasets:
- Event StoryLine Corpus (ESC)
- Richer Event Description (RED)
- MAVEN-ERE (a subset was used, following prior work)
Key Insights and Ideas
- Pruning is Key: The core idea of using a retriever to prune candidate event pairs before classification is highly effective. It directly addresses the quadratic complexity and the skewed data distribution.
- Retriever-Classifier Synergy: The retriever significantly enhances the performance of the cross-encoder, suggesting that identifying relevant pairs is a critical prerequisite for accurate relation classification.
- Coreference Chains Boost Performance: Incorporating event coreference chains as supplementary input further improves performance, especially for implicit event relations, by providing deeper contextual insights.
- "Wrapped" Encoding is Superior: Encoding event mentions by wrapping them with special markers (
<m>...</m>) within their sentences, as opposed to just using trigger words or graph-based methods, proved most effective for capturing contextual nuances.
Key Results
- Performance Improvement: The proposed two-stage framework, particularly with the S-BERT retriever and T5 classifier, achieved state-of-the-art performance on the evaluated datasets. For example, on ESC, the S-BERT + T5 combination achieved an F1 score of 90.8% on all event pairs.
- Efficiency Gains: The framework significantly reduces inference time. Figure 3 shows that their approach reduces inference time by approximately 70% compared to conventional pairwise methods, especially as the number of events per document increases.
- Skewed Distribution Alleviation: The method showed a significant increase in recall, particularly on datasets with many negative samples, indicating that the pruning strategy effectively alleviates the skewed distribution problem.
- Coreference Chains Impact: Adding coreference chains improved performance across the board, with RoBERTa retriever showing notable gains, suggesting its ability to leverage deep contextual insights.
Limitations and Future Work
The authors acknowledge two main limitations:
- Dataset Scope: The study was limited to three representative DERE datasets. Future work should explore the method's generalizability across more languages and diverse DERE datasets.
- Cross-Encoder Choice: While RoBERTa and T5 were used, more advanced or differently prompted cross-encoders might yield even better results.
Future work includes adapting the method to other Information Extraction (IE) tasks, such as entity relation extraction, and verifying its broader generalizability.
Summary updated