Abstract
Traditional Event Extraction (EE) models often rely on external syntactic parsers to capture structural relationships between triggers and arguments. However, this dependency introduces error propagation and computational overhead. This paper presents a "Soft Syntactic Reinforcement" (SSR) mechanism that leverages the latent syntactic knowledge already present within Pre-trained Language Models (PLMs). By training a simple linear transformation on a gold-standard syntax corpus, the model learns to "activate" structural dimensions without requiring external parsers at runtime. Our proposed Syntax-Reinforced Event (SRE) model achieves state-of-the-art results across multiple benchmarks, demonstrating significant improvements in recall and efficiency.
Introduction
Event Extraction (EE) involves identifying event triggers (actions) and their associated arguments (participants) within unstructured text. For example, in the sentence "An extremist detonated a bomb," "detonated" is the trigger, while "extremist" and "bomb" are arguments. Capturing the relationship between these elements often requires understanding the sentence's grammatical structure.
Historically, researchers have integrated external dependency parsers to provide this structural "skeleton." However, this "hard" integration suffers from two primary issues: error propagation from the parser to the extraction model, and redundancy, as modern PLMs already encode significant syntactic information.
To address these challenges, we propose a method that reinforces the syntax already latent within a PLM, avoiding the fragility of external tools during inference.
Methodology: Soft Syntactic Reinforcement
The core of our approach is the Syntax-Reinforced Event (SRE) model, which utilizes a novel Soft Syntactic Reinforcement (SSR) mechanism.
1. Syntactic Probing and Activation
Instead of parsing every sentence in the task dataset, we use a small, high-quality corpus (the Penn Tree Bank) to train a linear transformation. This transformation identifies the dimensions in the PLM's vector space that correspond to two key structural features:
- Syntactic Depth: The hierarchical level of a word relative to the sentence root.
- Syntactic Distance: The number of edges between two words in a dependency tree.
2. Model Architecture
The model learns transformation matrices ($B_{dep}$ and $B_{dist}$) that "activate" these latent syntactic properties. During the EE task, a "soft" gating mechanism—similar to an attention gate—fuses the original PLM representations with these reinforced, syntax-aware features. This allows the model to prioritize structural information when necessary without overwriting general semantic knowledge.
For decoding, we employ a Conditional Random Field (CRF) for sentence-level tasks and a BiLSTM for document-level context, ensuring the model captures both local and global dependencies.
Experimental Evaluation
We evaluated the SRE model across several diverse benchmarks, including general news (ACE05), cybersecurity (CASIE), medical reports (PHEE), and complex terrorist event reports (MUC-4).
Key Results
- State-of-the-Art Performance: The SRE model achieved an F1 score of 72.18% on ACE05 Trigger Classification and a significant 63.70% F1 on CASIE Argument Classification.
- Enhanced Recall: On the document-level MUC-4 dataset, the model saw a substantial boost in Recall (67.75%). This suggests that syntactic reinforcement is particularly effective at linking triggers to distant arguments in long, complex documents.
- Efficiency: Unlike traditional methods that require slow external parsers, the SSR module's pre-training takes only 3-5 hours, and its impact on inference time is negligible.
Discussion and Conclusion
Our findings suggest that syntax is latent, not absent, in modern language models. By helping the model "find" and prioritize its internal structural knowledge, we can achieve superior performance without the overhead of external parsers.
While the current work focuses on linear transformations and dependency trees, future research could explore non-linear mappings or other structural formats like constituency trees. Furthermore, domain-specific syntactic pre-training may further enhance performance in specialized fields. In conclusion, Soft Syntactic Reinforcement provides a robust, efficient, and effective path forward for structural language understanding tasks.
Summary updated