Introduction
Event Extraction (EE) is a fundamental task in Information Extraction that involves identifying triggers (words signaling an event) and their associated arguments (participants or context). While Pre-trained Language Models (PLMs) like BERT have significantly advanced the field, they often struggle with "ambiguous entanglement." Standard self-attention mechanisms look at sentences globally, which can dilute the focus on the specific syntactic neighborhood—the event field—that provides the most critical context for a trigger. Furthermore, existing models often treat multiple occurrences of the same trigger or argument as isolated events, failing to leverage redundant information to build a stronger representation.
The ProCE Framework
To address these challenges, the authors propose ProCE (Probabilistic reCoupling model enhanced Event extraction). This framework introduces a probabilistic approach to clarify and merge information within the event field.
ProCE is built on three key innovations:
- Event Field Clarification: Instead of relying solely on raw self-attention, the model applies a Gaussian prior to the attention scores. This bias is centered on the trigger or argument and covers its 1-hop syntactic neighbors identified via a dependency parser. This mathematically strengthens the focus on relevant local context.
- Event Field Recoupling: When a trigger (e.g., "attack") appears multiple times in a sentence, ProCE uses Gaussian Multiplication or Gaussian Mixture Models (GMM) to merge these individual fields. This allows the model to capture latent information shared between identical triggers, effectively performing "message passing" between them.
- Event Field Regularization: To ensure consistency, the authors introduce a loss function based on Wasserstein (WA) distance. This regularizes the output distributions of identical triggers or arguments, forcing the model to reach similar conclusions regardless of where the element appears in the sentence.
Experimental Evaluation
The model was evaluated on three major benchmarks: ACE 2005 (English), the gold standard for EE; OntoEvent, a large-scale dataset with over 100 event types; and FewEvent, specifically designed to test performance in data-scarce, few-shot settings.
ProCE treats syntactic context as a probabilistic region rather than a rigid graph, avoiding common issues like "over-smoothing" found in deep Graph Convolutional Networks (GCNs). Additionally, it is designed as a plug-and-play enhancement for the original BERT architecture, requiring no total overhaul of underlying parameters.
Results and Discussion
ProCE demonstrated significant improvements across all benchmarks. On ACE 2005, ProCE achieved an F1 score of 76.80 for Event Detection, a substantial jump from the baseline DMBERT's 70.87.
The most striking results appeared in low-resource settings. As shown in the table above, on the FewEvent dataset using only 10% of the training data, ProCE achieved a 47.13% F1 score, compared to just 34.93% for the baseline. This suggests that the probabilistic recoupling of redundant information is particularly effective when data is limited.
Conclusion and Future Work
ProCE successfully "recouples" the event field to provide more robust representations for Event Extraction. While the method currently relies on the accuracy of external syntactic dependency parsers, its success suggests potential for expansion into other tasks like Relation Extraction. Future work may also explore the computational overhead of calculating Wasserstein distances and GMMs on extremely large-scale datasets.
Summary updated