Recouple Event Field via Probabilistic Bias for Event Extraction

Xingyu Bai, Taiqiang Wu, Han Guo, Zhe Zhao, Xuefeng Yang, Jiayi Li, Weijie Liu, Qi Ju, Weigang Guo, Yujiu Yang

Introduction

Event Extraction (EE) is a fundamental task in Information Extraction that involves identifying triggers (words signaling an event) and their associated arguments (participants or context). While Pre-trained Language Models (PLMs) like BERT have significantly advanced the field, they often struggle with "ambiguous entanglement." Standard self-attention mechanisms look at sentences globally, which can dilute the focus on the specific syntactic neighborhood—the event field—that provides the most critical context for a trigger. Furthermore, existing models often treat multiple occurrences of the same trigger or argument as isolated events, failing to leverage redundant information to build a stronger representation.

The ProCE Framework

To address these challenges, the authors propose ProCE (Probabilistic reCoupling model enhanced Event extraction). This framework introduces a probabilistic approach to clarify and merge information within the event field.

The ProCE architecture overview, illustrating syntactic dependency parsing, self-attention with Gaussian bias, and the recoupling and regularization mechanisms.

ProCE is built on three key innovations:

  1. Event Field Clarification: Instead of relying solely on raw self-attention, the model applies a Gaussian prior to the attention scores. This bias is centered on the trigger or argument and covers its 1-hop syntactic neighbors identified via a dependency parser. This mathematically strengthens the focus on relevant local context.
  2. Event Field Recoupling: When a trigger (e.g., "attack") appears multiple times in a sentence, ProCE uses Gaussian Multiplication or Gaussian Mixture Models (GMM) to merge these individual fields. This allows the model to capture latent information shared between identical triggers, effectively performing "message passing" between them.
  3. Event Field Regularization: To ensure consistency, the authors introduce a loss function based on Wasserstein (WA) distance. This regularizes the output distributions of identical triggers or arguments, forcing the model to reach similar conclusions regardless of where the element appears in the sentence.

Experimental Evaluation

The model was evaluated on three major benchmarks: ACE 2005 (English), the gold standard for EE; OntoEvent, a large-scale dataset with over 100 event types; and FewEvent, specifically designed to test performance in data-scarce, few-shot settings.

ProCE treats syntactic context as a probabilistic region rather than a rigid graph, avoiding common issues like "over-smoothing" found in deep Graph Convolutional Networks (GCNs). Additionally, it is designed as a plug-and-play enhancement for the original BERT architecture, requiring no total overhaul of underlying parameters.

Results and Discussion

ProCE demonstrated significant improvements across all benchmarks. On ACE 2005, ProCE achieved an F1 score of 76.80 for Event Detection, a substantial jump from the baseline DMBERT's 70.87.

Performance comparison on the FewEvent dataset across different training data percentages, highlighting ProCE's dominance in low-resource settings.

The most striking results appeared in low-resource settings. As shown in the table above, on the FewEvent dataset using only 10% of the training data, ProCE achieved a 47.13% F1 score, compared to just 34.93% for the baseline. This suggests that the probabilistic recoupling of redundant information is particularly effective when data is limited.

Conclusion and Future Work

ProCE successfully "recouples" the event field to provide more robust representations for Event Extraction. While the method currently relies on the accuracy of external syntactic dependency parsers, its success suggests potential for expansion into other tasks like Relation Extraction. Future work may also explore the computational overhead of calculating Wasserstein distances and GMMs on extremely large-scale datasets.

Summary updated

Abstract

Event Extraction (EE), aiming to identify and classify event triggers and arguments from event mentions, has benefited from pre-trained language models (PLMs). However, existing PLM-based methods ignore the information of trigger/argument fields, which is crucial for understanding event schemas. To this end, we propose a Probabilistic reCoupling model enhanced Event extraction framework (ProCE). Specifically, we first model the syntactic-related event fields as probabilistic biases, to clarify the event fields from ambiguous entanglement. Furthermore, considering multiple occurrences of the same triggers/arguments in EE, we explore probabilistic interaction strategies among multiple fields of the same triggers/arguments, to recouple the corresponding clarified distributions and capture more latent information fields. Experiments on EE datasets demonstrate the effectiveness and generalization of our proposed approach.
BibTeX
@inproceedings{bai2023recouple,
  author = {Xingyu Bai and Taiqiang Wu and Han Guo and Zhe Zhao and Xuefeng Yang and Jiayi Li and Weijie Liu and Qi Ju and Weigang Guo and Yujiu Yang},
  title = {{Recouple Event Field via Probabilistic Bias for Event Extraction}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10096096},
}
Figure