Abstract
Event Argument Extraction (EAE) is a challenging task that involves identifying the roles of entities within an event. Traditional models often struggle with long-range dependencies between triggers and arguments and are prone to overfitting due to the small size of available datasets. We introduce SemSynGTN, a novel architecture that dynamically integrates syntactic and semantic sentence structures through Graph Transformer Networks (GTNs). Furthermore, we employ Information Bottleneck (IB) regularization to ensure the model focuses on task-relevant features. Our experiments on the ACE 2005 and TAC KBP 2016 benchmarks demonstrate that SemSynGTN achieves state-of-the-art results, particularly in complex linguistic scenarios.
Introduction
The goal of Event Argument Extraction (EAE) is to determine how entities relate to specific events in text. For example, in the sentence "The company was sold," an EAE system identifies "company" as the Object of a Transaction event. Despite its importance, the task faces two major hurdles:
- Structural Limitations: Syntactic dependency trees, while useful, often fail when arguments and event triggers are separated by many words. Purely semantic models capture context but may miss rigid structural constraints.
- The Generalization Gap: EAE datasets are notoriously small, causing deep learning models to overfit and perform poorly on unseen data.
Methodology: The SemSynGTN Model
SemSynGTN addresses these challenges by fusing multiple structural views of a sentence and applying information-theoretic regularization.
Dynamic Graph Construction
Instead of a static graph, the model constructs four distinct views:
- General Syntax: A standard dependency adjacency matrix.
- Task-Specific Syntax: Matrices tailored to the distance between words and the specific trigger/argument pair.
- Task-Specific Semantics: A matrix representing semantic importance, calculated via a query-key mechanism conditioned on the trigger and argument.
Graph Transformer Networks (GTNs)
The GTN acts as the model's core, treating the different graph views as edge types. It learns to multiply these matrices to identify "meta-paths"—multi-hop connections that combine syntactic and semantic information. This enables the model to find efficient paths between distant words that neither structure could identify alone.
Information Bottleneck (IB) Regularization
To prevent the model from memorizing noise in small datasets, the authors treat the GTN as an information bottleneck. Using Mutual Information Neural Estimation (MINE), they minimize the mutual information between the input encoding and the GTN's output. This forces the model to discard "nuisance" information and retain only the "effective" information necessary for accurate prediction.
Experimental Evaluation
The model was evaluated on two gold-standard benchmarks: ACE 2005 (599 documents) and TAC KBP 2016 (used for testing transfer performance).
State-of-the-Art Results
SemSynGTN outperformed all previous models on both benchmarks. On the ACE 2005 dataset, the BERT-based version of SemSynGTN achieved an F1 score of 61.9, a significant improvement over the previous state-of-the-art model, HMEAE (59.3).
The model also showed strong performance on TAC KBP 2016, achieving an F1 score of 28.5 compared to HMEAE's 25.1. Notably, SemSynGTN demonstrated superior robustness in cases where the distance between the trigger and argument exceeded 10 words.
Ablation Analysis
Ablation studies confirmed the necessity of both the GTN and IB components. Removing the GTN component or the IB regularization led to a measurable drop in performance, proving that both structural fusion and information-theoretic regularization are essential for the model's success.
Conclusion
SemSynGTN represents a significant advancement in Event Argument Extraction by effectively combining syntactic and semantic structures. The use of Graph Transformer Networks to learn task-specific meta-paths and Information Bottleneck regularization to handle small-data regimes provides a powerful framework for information extraction. Future research will explore extending this approach to other tasks like relation extraction and improving the efficiency of mutual information estimation.
Summary updated