Graph Transformer Networks with Syntactic and Semantic Structures for Event Argument Extraction

Amir Pouran Ben Veyseh, Tuan Ngo Nguyen, Thien Huu Nguyen

Abstract

Event Argument Extraction (EAE) is a challenging task that involves identifying the roles of entities within an event. Traditional models often struggle with long-range dependencies between triggers and arguments and are prone to overfitting due to the small size of available datasets. We introduce SemSynGTN, a novel architecture that dynamically integrates syntactic and semantic sentence structures through Graph Transformer Networks (GTNs). Furthermore, we employ Information Bottleneck (IB) regularization to ensure the model focuses on task-relevant features. Our experiments on the ACE 2005 and TAC KBP 2016 benchmarks demonstrate that SemSynGTN achieves state-of-the-art results, particularly in complex linguistic scenarios.

Introduction

The goal of Event Argument Extraction (EAE) is to determine how entities relate to specific events in text. For example, in the sentence "The company was sold," an EAE system identifies "company" as the Object of a Transaction event. Despite its importance, the task faces two major hurdles:

  1. Structural Limitations: Syntactic dependency trees, while useful, often fail when arguments and event triggers are separated by many words. Purely semantic models capture context but may miss rigid structural constraints.
  2. The Generalization Gap: EAE datasets are notoriously small, causing deep learning models to overfit and perform poorly on unseen data.

Methodology: The SemSynGTN Model

SemSynGTN addresses these challenges by fusing multiple structural views of a sentence and applying information-theoretic regularization.

Dynamic Graph Construction

Instead of a static graph, the model constructs four distinct views:

  • General Syntax: A standard dependency adjacency matrix.
  • Task-Specific Syntax: Matrices tailored to the distance between words and the specific trigger/argument pair.
  • Task-Specific Semantics: A matrix representing semantic importance, calculated via a query-key mechanism conditioned on the trigger and argument.

Graph Transformer Networks (GTNs)

The GTN acts as the model's core, treating the different graph views as edge types. It learns to multiply these matrices to identify "meta-paths"—multi-hop connections that combine syntactic and semantic information. This enables the model to find efficient paths between distant words that neither structure could identify alone.

Information Bottleneck (IB) Regularization

To prevent the model from memorizing noise in small datasets, the authors treat the GTN as an information bottleneck. Using Mutual Information Neural Estimation (MINE), they minimize the mutual information between the input encoding and the GTN's output. This forces the model to discard "nuisance" information and retain only the "effective" information necessary for accurate prediction.

Experimental Evaluation

The model was evaluated on two gold-standard benchmarks: ACE 2005 (599 documents) and TAC KBP 2016 (used for testing transfer performance).

State-of-the-Art Results

SemSynGTN outperformed all previous models on both benchmarks. On the ACE 2005 dataset, the BERT-based version of SemSynGTN achieved an F1 score of 61.9, a significant improvement over the previous state-of-the-art model, HMEAE (59.3).

Performance comparison on the ACE 2005 dataset.

The model also showed strong performance on TAC KBP 2016, achieving an F1 score of 28.5 compared to HMEAE's 25.1. Notably, SemSynGTN demonstrated superior robustness in cases where the distance between the trigger and argument exceeded 10 words.

Ablation Analysis

Ablation studies confirmed the necessity of both the GTN and IB components. Removing the GTN component or the IB regularization led to a measurable drop in performance, proving that both structural fusion and information-theoretic regularization are essential for the model's success.

Ablation study showing the impact of GTN and IB components.

Conclusion

SemSynGTN represents a significant advancement in Event Argument Extraction by effectively combining syntactic and semantic structures. The use of Graph Transformer Networks to learn task-specific meta-paths and Information Bottleneck regularization to handle small-data regimes provides a powerful framework for information extraction. Future research will explore extending this approach to other tasks like relation extraction and improving the efficiency of mutual information estimation.

Summary updated

Abstract

The goal of Event Argument Extraction (EAE) is to find the role of each entity mention for a given event trigger word. It has been shown in the previous works that the syntactic structures of the sentences are helpful for the deep learning models for EAE. However, a major problem in such prior works is that they fail to exploit the semantic structures of the sentences to induce effective representations for EAE. Consequently, in this work, we propose a novel model for EAE that exploits both syntactic and semantic structures of the sentences with the Graph Transformer Networks (GTNs) to learn more effective sentence structures for EAE. In addition, we introduce a novel inductive bias based on information bottleneck to improve generalization of the EAE models. Extensive experiments are performed to demonstrate the benefits of the proposed model, leading to state-of-the-art performance for EAE on standard datasets.
BibTeX
@inproceedings{veyseh2020graph,
  author = {Amir Pouran Ben Veyseh and Tuan Ngo Nguyen and Thien Huu Nguyen},
  title = {{Graph Transformer Networks with Syntactic and Semantic Structures for Event Argument Extraction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.326},
}
Figure