Presented at NAACL 2025
Abstract
Document-level event extraction (DEE) is a critical task in information extraction, yet existing models often fail when a single document contains multiple similar ("homogeneous") events. This paper introduces EPAL (Event-specific Probe and Argument Library), a framework designed to overcome "event confusion" through the use of entity-based probes and a differential optimization strategy. Evaluated on the ChFinAnn and DuEE-Fin datasets, EPAL demonstrates significant improvements in recall and F1-score, particularly for multi-event extraction tasks.
The Challenge of Homogeneous Events
In Document-Level Event Extraction (DEE), the goal is to identify event types and their corresponding arguments across an entire text. While current models are effective at identifying single events, they struggle with homogeneous events—multiple instances of the same event type that share many "public" arguments but differ in a few "private" ones.
For example, a financial report might list several different equity pledges by the same company on different dates for different amounts. Existing models often suffer from "event confusion," either merging these into one record or missing the subtle differences that separate them. This leads to high precision but very low recall in multi-event documents.
The EPAL Framework
The authors propose EPAL to address these limitations through three main innovations:
- Event-specific Probes: Instead of a fixed number of event slots, the model treats every entity in the document as a potential "probe." Each probe's job is to "sniff" the document to see if it belongs to a specific event instance.
- Argument Library: For every probe, the model constructs a library of all other entities. It uses a Multi-Head Attention mechanism to determine which entities in the library fill specific roles (like "Pledger" or "Amount") for the event that the probe has identified.
- Differential Optimization (Probe-Label Alignment): Training a model to map a set of predicted events to ground-truth labels is often unstable. EPAL uses a novel alignment algorithm that differentially optimizes the embeddings of probes and arguments, ensuring that each ground-truth event is "claimed" by the most accurate probe during training.
To further refine the model, two auxiliary losses are added: a Role Contrastive Loss ($L_{rc}$) to distinguish between different arguments playing the same role in different events, and a Probe Consistent Loss ($L_{pc}$) to ensure consistent representations for probe categories in the latent space.
Experimental Evaluation
The model was evaluated on two major Chinese financial event extraction benchmarks: ChFinAnn (32,040 documents) and DuEE-Fin (11,900 documents). Both datasets contain a high density of multi-event samples (29% and 33%, respectively).
EPAL outperformed six state-of-the-art baselines, including ProCNet and ReDEE:
- ChFinAnn: Achieved an F1-score of 83.4, a 1.5-point improvement over the previous best. Crucially, its Multi-event F1 (F1(M)) reached 76.6, outperforming the next best model by 2.5 points.
- DuEE-Fin: Achieved an F1-score of 76.4.
- Recall Boost: The model showed a 4-8% improvement in recall for multi-event documents, proving its superior ability to discover "hidden" events that other models overlook.
Key Insights
A primary realization of this work is that private arguments (entities appearing in only one event record) serve as essential anchors. By initializing probes based on these entities, the model naturally learns to separate overlapping events. Furthermore, the results confirm that while precision is often high in DEE, recall is the primary challenge. EPAL targets this "missing event" problem with an aggressive discovery strategy balanced by stable optimization.
Limitations and Future Work
Despite its success, EPAL faces some challenges. The use of every entity as a probe introduces computational redundancy, as many probes ultimately detect no event. Future work may involve filtering mechanisms to screen entities beforehand. Additionally, the model occasionally confuses numerical arguments in similar contexts, suggesting a need for improved "number sense" in future iterations. Finally, the authors note that performance on very long documents could be enhanced through more complex cross-paragraph reasoning, such as heterogeneous graphs.
Summary updated